Serialization

Saving and loading SameDiff graphs in FlatBuffers format

SameDiff graphs — including their structure, variables, constants, and trained parameters — can be saved to and loaded from files using the FlatBuffers binary format. This enables model deployment, checkpointing during training, and sharing trained models.

Saving a SameDiff Graph

save()

Save the complete graph including all variables, constants, and parameter values:

import org.nd4j.autodiff.samediff.SameDiff;
import java.io.File;

// Save with updater state (for resuming training)
sd.save(new File("model.fb"), true);

// Save without updater state (smaller file, for inference only)
sd.save(new File("model.fb"), false);

The second argument controls whether to include the updater state (momentum buffers, adaptive learning rate accumulators, etc.). Include it if you plan to resume training; omit it for inference-only deployment.

asFlatGraph()

Convert the graph to a FlatBuffers byte buffer in memory (useful for embedding in other formats or sending over the network):

import java.nio.ByteBuffer;

ByteBuffer buffer = sd.asFlatGraph(true);  // true = include updater state

asFlatPrint()

Get a human-readable string representation of the graph (for debugging):

String graphString = sd.asFlatPrint();
System.out.println(graphString);

This prints the graph structure, variable names, shapes, and operation types.

Loading a SameDiff Graph

load()

Load a previously saved graph:

// Load with updater state (for resuming training)
SameDiff sd = SameDiff.load(new File("model.fb"), true);

// Load without updater state (for inference)
SameDiff sd = SameDiff.load(new File("model.fb"), false);

fromFlatGraph()

Load from a FlatBuffers byte buffer:

import org.nd4j.autodiff.samediff.SameDiff;
import java.nio.ByteBuffer;

SameDiff sd = SameDiff.fromFlatGraph(byteBuffer);

What Gets Saved

Component	Saved	Notes
Graph structure (ops, connections)	Always	The computation graph topology
VARIABLE values (weights, biases)	Always	Trainable parameters
CONSTANT values	Always	Non-trainable stored values
PLACEHOLDER definitions	Always	Shape and type info (not values)
ARRAY definitions	Always	Shape and type info (not values — computed at runtime)
Updater state	Optional	Momentum/adaptive rate buffers. Only if saveUpdater=true
TrainingConfig	With updater	Optimizer settings

Training Checkpoints

Save periodic checkpoints during training for recovery:

for (int epoch = 0; epoch < numEpochs; epoch++) {
    sd.fit(trainIter, 1);
    trainIter.reset();

    // Save checkpoint every 5 epochs (with updater state)
    if (epoch % 5 == 0) {
        sd.save(new File("checkpoint_epoch_" + epoch + ".fb"), true);
    }
}

// Save final model (without updater state — smaller)
sd.save(new File("final_model.fb"), false);

Resume training from a checkpoint:

SameDiff sd = SameDiff.load(new File("checkpoint_epoch_10.fb"), true);

// Re-set training config if needed
sd.setTrainingConfig(TrainingConfig.builder()
    .updater(new Adam(1e-4))    // can change learning rate
    .dataSetFeatureMapping("input")
    .dataSetLabelMapping("label")
    .build());

// Continue training
sd.fit(trainIter, remainingEpochs);

Interop with Model Import

SameDiff is also the target format for model import from other frameworks. When you import a TensorFlow or ONNX model, the result is a SameDiff graph:

// TensorFlow import
import org.nd4j.imports.graphmapper.tf.TFGraphMapper;
SameDiff sd = TFGraphMapper.importGraph(new File("frozen_model.pb"));

// ONNX import
import org.nd4j.imports.graphmapper.onnx.OnnxGraphMapper;
SameDiff sd = OnnxGraphMapper.importGraph(new File("model.onnx"));

// Run inference on the imported model
Map<String, INDArray> placeholders = new HashMap<>();
placeholders.put("input:0", inputData);
INDArray output = sd.outputSingle(placeholders, "output:0");

// Save as FlatBuffers for faster subsequent loading
sd.save(new File("converted_model.fb"), false);

This workflow (import once, save as FlatBuffers, load FlatBuffers for serving) avoids repeated parsing of the original model format.

File Format Details

SameDiff uses FlatBuffers as its serialization format:

• Binary format: Compact, fast to serialize/deserialize

• No schema evolution issues: Forward and backward compatible

• Zero-copy reads: FlatBuffers can be read directly from the buffer without unpacking

• Cross-platform: Same file works on any OS/architecture

Typical file sizes depend on model complexity:

• Simple MLP (784→256→10): ~1-2 MB

• ResNet-18: ~45 MB

• Large transformer: 100+ MB

Best Practices

1. Save without updater state for deployment — reduces file size significantly (updater state can be 2-3x the model parameters for Adam)

2. Save with updater state for checkpoints — allows seamless training resumption

3. Convert imported models to FlatBuffers — much faster to load than parsing TF/ONNX format each time

4. Version your saved models — include epoch/date in filenames for traceability

5. Verify after loading — run a sample inference to confirm the loaded model produces expected results

// Quick verification after loading
SameDiff sd = SameDiff.load(new File("model.fb"), false);
INDArray testInput = Nd4j.rand(DataType.FLOAT, 1, inputSize);
Map<String, INDArray> ph = Collections.singletonMap("input", testInput);
INDArray output = sd.outputSingle(ph, "predictions");
System.out.println("Output shape: " + Arrays.toString(output.shape()));
System.out.println("Output: " + output);

---

SDNB and SDZ Formats (ADR 0035)

The original .fb FlatBuffers format has a hard 2 GB ceiling imposed by the FlatBuffers 32-bit size field. Two new container formats overcome this limit while also improving deployment ergonomics.

Format Overview

Format	Extension	Description	Best For
SDNB	.sdnb	Single-file binary with section header, manifest, graph, and appended arrays	Training checkpoints, highest performance I/O
SDZ	.sdz	Standard ZIP archive containing one or more .sdnb shards	Deployment, single-file distribution, compressed storage
Sharded SDNB	.shard0-of-N.sdnb …	Multiple .sdnb files, one per shard	Very large models (multi-hundred GB weights)

SameDiff.load() automatically detects all three formats by inspecting the file magic bytes and directory entries, so existing code that calls SameDiff.load() does not need to change.

SDNB Format

SDNB (SameDiff Native Binary) is a section-based binary container. Its file structure is:

MAGIC_BYTES (4 bytes: "SDNB")
VERSION     (4 bytes)
MANIFEST_OFFSET (8 bytes)
MANIFEST_LENGTH (8 bytes)
METADATA_OFFSET (8 bytes)
[FLATBUFFER_GRAPH_DATA]
[APPENDED_ARRAYS_DATA]
[SERIALIZED_MANIFEST]

Arrays are appended after the FlatBuffers graph region, bypassing the 2 GB limit. A manifest records the byte offset and length of every array so the loader can seek directly to each one without scanning the file.

import org.nd4j.autodiff.samediff.serde.SameDiffSerializer;
import java.io.File;
import java.util.Map;

// Save as a single .sdnb file (no sharding)
SameDiffSerializer.save(sd, new File("model.sdnb"), false, null);

// Save with metadata
Map<String, String> meta = new LinkedHashMap<>();
meta.put("model.name", "my-classifier");
meta.put("model.version", "1.2.0");
meta.put("training.epochs", "100");
SameDiffSerializer.save(sd, new File("model.sdnb"), true, meta);

// Load
SameDiff loaded = SameDiffSerializer.load(new File("model.sdnb"), false);

SDZ Format

SDZ packages one or more .sdnb shards into a standard ZIP archive. This gives you a single file you can inspect with any ZIP tool (unzip -l model.sdz) and distribute without managing shard files separately. Compression typically reduces file size by 30–50%.

import org.nd4j.autodiff.samediff.serde.SDZSerializer;
import java.io.File;
import java.util.Map;

// Save as a single .sdz file (handles sharding automatically)
Map<String, String> meta = Map.of(
    "model.name", "bert-base",
    "model.version", "2.0"
);
SDZSerializer.save(sd, new File("model.sdz"), false, meta);

// Load — automatically extracts to a temp dir and cleans up after
SameDiff loaded = SDZSerializer.load(new File("model.sdz"), false);

Internally SDZSerializer.save() calls SameDiffSerializer.saveAutoShard() to create the SDNB shards, then compresses them into the ZIP. SDZSerializer.load() extracts all shards to a temp directory, loads them in order, and deletes the temp directory on completion.

Sharding for Large Models (>2 GB)

When the total model weight exceeds what a single SDNB file can hold efficiently, sharding distributes variables across multiple files. Shard 0 always contains the graph structure; subsequent shards hold variable data.

// Let the library decide the shard count based on model size
// (maximum 1 GB per shard)
SameDiffSerializer.saveAutoShard(
    sd,
    new File("large_model"),   // base name — produces large_model.shard0-of-N.sdnb, etc.
    false,
    meta
);

// Or specify an exact shard count
SameDiffSerializer.saveSharded(
    sd,
    new File("large_model"),
    false,
    8,        // 8 shards
    meta
);

// Load sharded model — auto-detected by SameDiff.load()
SameDiff loaded = SameDiff.load(new File("large_model"), false);

// Or load sharded explicitly
SameDiff loaded = SameDiffSerializer.loadSharded(new File("large_model"), false);

For models intended for single-file distribution, prefer saving as SDZ: it shards internally and then bundles everything into one .sdz file.

Format Auto-Detection

SameDiff.load() probes the file and automatically selects the right loader:

// Works for .fb (legacy FlatBuffers), .sdnb, .sdz, and sharded SDNB base paths
SameDiff sd = SameDiff.load(new File("model.sdz"), false);
SameDiff sd = SameDiff.load(new File("model.sdnb"), false);
SameDiff sd = SameDiff.load(new File("model.fb"), false);       // legacy
SameDiff sd = SameDiff.load(new File("large_model"), false);    // sharded base name

Metadata

Both SDNB and SDZ support an extensible key-value metadata map (similar to GGUF). Metadata can be added after the model is saved without reserializing the weight tensors. Standardized keys include:

Key	Meaning
model.name	Human-readable model name
model.version	Semantic version string
training.epochs	Number of training epochs completed
training.dataset	Dataset used for training
nd4j.format.version	Format version (set automatically)

When to Use Each Format

Scenario	Recommended Format
Training checkpoints (fast save/load)	SDNB
Resuming training	SDNB with saveUpdaterState=true
Deployment / single-file distribution	SDZ
Models larger than 2 GB	SDZ (auto-shards internally) or sharded SDNB
Inspecting contents with standard tools	SDZ (unzip -l model.sdz)
Migrating an existing .fb model	Load .fb, save as .sdz

Backward Compatibility

Existing .fb files continue to load without any changes — the auto-detection path checks for the legacy format first. No migration is required for models already in production.