Serialization
Saving and loading INDArrays in binary, text, NumPy, and ByteBuffer formats
Serialization
ND4J provides several serialization formats for INDArray instances. Choosing the right one depends on your priorities: raw throughput, human-readable output, or interoperability with Python/NumPy pipelines.
Binary (Nd4j)
Nd4j.write
Nd4j.read
Fast file persistence
ByteBuffer (BinarySerde)
BinarySerde.toByteBuffer
BinarySerde.toArray
In-memory / IPC transfer
Text
Nd4j.writeTxt
Nd4j.readTxt
Debugging, inspection
NumPy text (CSV)
—
Nd4j.readNumpy
Import CSV/NumPy text
NumPy binary (.npy)
—
Nd4j.createFromNpyFile
Python interoperability
1. Binary Format
The binary format is the most compact and fastest option for disk persistence. It serializes the full array — shape, data type, order, and element values — into a compact byte stream using Java's DataOutputStream / DataInputStream.
Write:
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.factory.Nd4j;
import java.io.*;
INDArray arr = Nd4j.linspace(1, 10, 10).reshape(2, 5);
try (DataOutputStream out = new DataOutputStream(
new BufferedOutputStream(new FileOutputStream("array.bin")))) {
Nd4j.write(arr, out);
}Read:
The deserialized array is identical to the original: same shape, same data type, same element ordering. Wrapping the underlying stream in a BufferedOutputStream / BufferedInputStream is recommended for large arrays to avoid excessive system-call overhead.
When to use: File checkpoints between JVM sessions, persisting intermediate computation results, or anywhere you need the smallest on-disk footprint with the fastest read-back time.
2. ByteBuffer Format
BinarySerde serializes an INDArray into a java.nio.ByteBuffer. When the array's underlying data is already stored in a direct (off-heap) buffer, this conversion can avoid a data copy entirely, making it well-suited for inter-process communication, shared-memory transfers, or handing data off to a native library.
Maven dependency — BinarySerde lives in the nd4j-serde module:
Write to ByteBuffer:
Read from ByteBuffer:
Writing / reading via a file channel (useful for memory-mapped files):
When to use: In-memory handoffs between components running in the same JVM, native-bridge scenarios, or when using memory-mapped files for very large arrays. The zero-copy property only applies when the array's data buffer is already in a compatible direct-memory layout; otherwise a copy is made transparently.
3. Text Format
The text format writes each element as a human-readable floating-point number. Rows are delimited by newlines and columns by commas (the exact delimiter is locale-independent).
Write:
Read:
Example output for a 2x3 array:
Text serialization round-trips correctly for the default float precision, but be aware that the conversion to decimal representation and back introduces a tiny floating-point rounding error. For double-precision arrays where bit-exact round-tripping matters, prefer the binary format.
When to use: Spot-checking intermediate values during development, sharing arrays with non-Java tooling that can read CSV, or generating test fixtures that should be human-auditable.
4. NumPy Text (CSV) Format
Nd4j.readNumpy reads a plain-text file whose rows are whitespace- or comma-separated numbers — the same format produced by NumPy's numpy.savetxt and by many CSV export tools.
The result is always a 2D row-major INDArray. The delimiter must match exactly what was used during export. Common choices are "," for CSV and " " for space-separated files.
Note: Nd4j.writeNumpy (which wrote a compatible text file) was deprecated in earlier releases. For writing, use Nd4j.writeTxt or the NumPy binary format described below.
When to use: Importing datasets or weights that were exported from a Python/NumPy workflow using numpy.savetxt or any CSV-producing tool.
5. NumPy Binary (.npy) Format
.npy) FormatThe .npy binary format is the standard serialization format for single NumPy arrays. It stores the array's dtype, shape, byte order, and raw element data in a compact binary file that NumPy can read and write natively.
Python side — save a .npy file:
Java side — load the .npy file:
ND4J reads the header to determine the shape and dtype automatically — no manual configuration is needed. The returned array preserves the original dtype (e.g., float32 maps to DataType.FLOAT, float64 maps to DataType.DOUBLE).
Saving from Java for NumPy to read:
When to use: Any time you need to exchange arrays between a Java/DL4J pipeline and a Python/NumPy/PyTorch/TensorFlow pipeline. The .npy format is the most reliable interoperability path because it preserves dtype, shape, and byte order without ambiguity.
6. Additional Serialization Back-ends (nd4j-serde)
The nd4j-serde module provides adapters for other serialization ecosystems:
nd4j-serde/nd4j-jackson
Jackson JsonSerializer / JsonDeserializer for INDArray — embeds arrays as Base64-encoded binary inside JSON documents
nd4j-serde/nd4j-kryo
Kryo serializer for INDArray — useful in Apache Spark or other distributed frameworks that use Kryo as their wire format
nd4j-serde/nd4j-aeron
Aeron-compatible serialization for low-latency messaging
nd4j-serde/nd4j-camel-routes
Apache Camel data-format adapter
The Jackson integration is particularly common when embedding arrays in REST API payloads:
Choosing a Format
Use binary (Nd4j.write / Nd4j.read) when:
You need maximum speed and minimum file size for JVM-to-JVM persistence.
You are checkpointing intermediate results in a training loop.
Portability outside the JVM is not required.
Use BinarySerde when:
You are passing arrays between threads or processes within the same machine.
You want zero-copy semantics or need to work with memory-mapped files.
You are integrating with a native library through a
ByteBufferAPI.
Use text (Nd4j.writeTxt / Nd4j.readTxt) when:
You want to visually inspect values in a text editor or log file.
You are writing test fixtures that need to be human-readable and diffable.
Absolute bit-exact round-tripping is not required.
Use NumPy text (Nd4j.readNumpy) when:
You are importing data that was exported by
numpy.savetxtor another CSV tool.The producing system cannot write
.npybinary files.
Use NumPy binary (Nd4j.createFromNpyFile / Nd4j.writeAsNumpy) when:
You are integrating with a Python-based ecosystem (NumPy, pandas, PyTorch, TensorFlow).
You need dtype and shape metadata to be preserved automatically.
You want the most portable and widely supported binary array format available.
Complete Example
The following snippet demonstrates all formats in a single class:
Last updated
Was this helpful?