> For the complete documentation index, see [llms.txt](https://deeplearning4j.konduit.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://deeplearning4j.konduit.ai/en-1.0.0-rewrite/nd4j/overview-2/serialization.md). # Serialization SameDiff graphs — including their structure, variables, constants, and trained parameters — can be saved to and loaded from files using the FlatBuffers binary format. This enables model deployment, checkpointing during training, and sharing trained models. ## Saving a SameDiff Graph ### save() Save the complete graph including all variables, constants, and parameter values: ```java import org.nd4j.autodiff.samediff.SameDiff; import java.io.File; // Save with updater state (for resuming training) sd.save(new File("model.fb"), true); // Save without updater state (smaller file, for inference only) sd.save(new File("model.fb"), false); ``` The second argument controls whether to include the updater state (momentum buffers, adaptive learning rate accumulators, etc.). Include it if you plan to resume training; omit it for inference-only deployment. ### asFlatGraph() Convert the graph to a FlatBuffers byte buffer in memory (useful for embedding in other formats or sending over the network): ```java import java.nio.ByteBuffer; ByteBuffer buffer = sd.asFlatGraph(true); // true = include updater state ``` ### asFlatPrint() Get a human-readable string representation of the graph (for debugging): ```java String graphString = sd.asFlatPrint(); System.out.println(graphString); ``` This prints the graph structure, variable names, shapes, and operation types. ## Loading a SameDiff Graph ### load() Load a previously saved graph: ```java // Load with updater state (for resuming training) SameDiff sd = SameDiff.load(new File("model.fb"), true); // Load without updater state (for inference) SameDiff sd = SameDiff.load(new File("model.fb"), false); ``` ### fromFlatGraph() Load from a FlatBuffers byte buffer: ```java import org.nd4j.autodiff.samediff.SameDiff; import java.nio.ByteBuffer; SameDiff sd = SameDiff.fromFlatGraph(byteBuffer); ``` ## What Gets Saved | Component | Saved | Notes | | ---------------------------------- | ------------ | ---------------------------------------------------------- | | Graph structure (ops, connections) | Always | The computation graph topology | | VARIABLE values (weights, biases) | Always | Trainable parameters | | CONSTANT values | Always | Non-trainable stored values | | PLACEHOLDER definitions | Always | Shape and type info (not values) | | ARRAY definitions | Always | Shape and type info (not values — computed at runtime) | | Updater state | Optional | Momentum/adaptive rate buffers. Only if `saveUpdater=true` | | TrainingConfig | With updater | Optimizer settings | ## Training Checkpoints Save periodic checkpoints during training for recovery: ```java for (int epoch = 0; epoch < numEpochs; epoch++) { sd.fit(trainIter, 1); trainIter.reset(); // Save checkpoint every 5 epochs (with updater state) if (epoch % 5 == 0) { sd.save(new File("checkpoint_epoch_" + epoch + ".fb"), true); } } // Save final model (without updater state — smaller) sd.save(new File("final_model.fb"), false); ``` Resume training from a checkpoint: ```java SameDiff sd = SameDiff.load(new File("checkpoint_epoch_10.fb"), true); // Re-set training config if needed sd.setTrainingConfig(TrainingConfig.builder() .updater(new Adam(1e-4)) // can change learning rate .dataSetFeatureMapping("input") .dataSetLabelMapping("label") .build()); // Continue training sd.fit(trainIter, remainingEpochs); ``` ## Interop with Model Import SameDiff is also the target format for model import from other frameworks. When you import a TensorFlow or ONNX model, the result is a `SameDiff` graph: ```java // TensorFlow import import org.nd4j.imports.graphmapper.tf.TFGraphMapper; SameDiff sd = TFGraphMapper.importGraph(new File("frozen_model.pb")); // ONNX import import org.nd4j.imports.graphmapper.onnx.OnnxGraphMapper; SameDiff sd = OnnxGraphMapper.importGraph(new File("model.onnx")); // Run inference on the imported model Map placeholders = new HashMap<>(); placeholders.put("input:0", inputData); INDArray output = sd.outputSingle(placeholders, "output:0"); // Save as FlatBuffers for faster subsequent loading sd.save(new File("converted_model.fb"), false); ``` This workflow (import once, save as FlatBuffers, load FlatBuffers for serving) avoids repeated parsing of the original model format. ## File Format Details SameDiff uses [FlatBuffers](https://google.github.io/flatbuffers/) as its serialization format: * **Binary format**: Compact, fast to serialize/deserialize * **No schema evolution issues**: Forward and backward compatible * **Zero-copy reads**: FlatBuffers can be read directly from the buffer without unpacking * **Cross-platform**: Same file works on any OS/architecture Typical file sizes depend on model complexity: * Simple MLP (784→256→10): \~1-2 MB * ResNet-18: \~45 MB * Large transformer: 100+ MB ## Best Practices 1. **Save without updater state for deployment** — reduces file size significantly (updater state can be 2-3x the model parameters for Adam) 2. **Save with updater state for checkpoints** — allows seamless training resumption 3. **Convert imported models to FlatBuffers** — much faster to load than parsing TF/ONNX format each time 4. **Version your saved models** — include epoch/date in filenames for traceability 5. **Verify after loading** — run a sample inference to confirm the loaded model produces expected results ```java // Quick verification after loading SameDiff sd = SameDiff.load(new File("model.fb"), false); INDArray testInput = Nd4j.rand(DataType.FLOAT, 1, inputSize); Map ph = Collections.singletonMap("input", testInput); INDArray output = sd.outputSingle(ph, "predictions"); System.out.println("Output shape: " + Arrays.toString(output.shape())); System.out.println("Output: " + output); ``` *** ## SDNB and SDZ Formats (ADR 0035) The original `.fb` FlatBuffers format has a hard 2 GB ceiling imposed by the FlatBuffers 32-bit size field. Two new container formats overcome this limit while also improving deployment ergonomics. ### Format Overview | Format | Extension | Description | Best For | | ------------ | --------------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------- | | SDNB | `.sdnb` | Single-file binary with section header, manifest, graph, and appended arrays | Training checkpoints, highest performance I/O | | SDZ | `.sdz` | Standard ZIP archive containing one or more `.sdnb` shards | Deployment, single-file distribution, compressed storage | | Sharded SDNB | `.shard0-of-N.sdnb` … | Multiple `.sdnb` files, one per shard | Very large models (multi-hundred GB weights) | `SameDiff.load()` automatically detects all three formats by inspecting the file magic bytes and directory entries, so existing code that calls `SameDiff.load()` does not need to change. ### SDNB Format SDNB (SameDiff Native Binary) is a section-based binary container. Its file structure is: ``` MAGIC_BYTES (4 bytes: "SDNB") VERSION (4 bytes) MANIFEST_OFFSET (8 bytes) MANIFEST_LENGTH (8 bytes) METADATA_OFFSET (8 bytes) [FLATBUFFER_GRAPH_DATA] [APPENDED_ARRAYS_DATA] [SERIALIZED_MANIFEST] ``` Arrays are appended after the FlatBuffers graph region, bypassing the 2 GB limit. A manifest records the byte offset and length of every array so the loader can seek directly to each one without scanning the file. ```java import org.nd4j.autodiff.samediff.serde.SameDiffSerializer; import java.io.File; import java.util.Map; // Save as a single .sdnb file (no sharding) SameDiffSerializer.save(sd, new File("model.sdnb"), false, null); // Save with metadata Map meta = new LinkedHashMap<>(); meta.put("model.name", "my-classifier"); meta.put("model.version", "1.2.0"); meta.put("training.epochs", "100"); SameDiffSerializer.save(sd, new File("model.sdnb"), true, meta); // Load SameDiff loaded = SameDiffSerializer.load(new File("model.sdnb"), false); ``` ### SDZ Format SDZ packages one or more `.sdnb` shards into a standard ZIP archive. This gives you a single file you can inspect with any ZIP tool (`unzip -l model.sdz`) and distribute without managing shard files separately. Compression typically reduces file size by 30–50%. ```java import org.nd4j.autodiff.samediff.serde.SDZSerializer; import java.io.File; import java.util.Map; // Save as a single .sdz file (handles sharding automatically) Map meta = Map.of( "model.name", "bert-base", "model.version", "2.0" ); SDZSerializer.save(sd, new File("model.sdz"), false, meta); // Load — automatically extracts to a temp dir and cleans up after SameDiff loaded = SDZSerializer.load(new File("model.sdz"), false); ``` Internally `SDZSerializer.save()` calls `SameDiffSerializer.saveAutoShard()` to create the SDNB shards, then compresses them into the ZIP. `SDZSerializer.load()` extracts all shards to a temp directory, loads them in order, and deletes the temp directory on completion. ### Sharding for Large Models (>2 GB) When the total model weight exceeds what a single SDNB file can hold efficiently, sharding distributes variables across multiple files. Shard 0 always contains the graph structure; subsequent shards hold variable data. ```java // Let the library decide the shard count based on model size // (maximum 1 GB per shard) SameDiffSerializer.saveAutoShard( sd, new File("large_model"), // base name — produces large_model.shard0-of-N.sdnb, etc. false, meta ); // Or specify an exact shard count SameDiffSerializer.saveSharded( sd, new File("large_model"), false, 8, // 8 shards meta ); // Load sharded model — auto-detected by SameDiff.load() SameDiff loaded = SameDiff.load(new File("large_model"), false); // Or load sharded explicitly SameDiff loaded = SameDiffSerializer.loadSharded(new File("large_model"), false); ``` For models intended for single-file distribution, prefer saving as SDZ: it shards internally and then bundles everything into one `.sdz` file. ### Format Auto-Detection `SameDiff.load()` probes the file and automatically selects the right loader: ```java // Works for .fb (legacy FlatBuffers), .sdnb, .sdz, and sharded SDNB base paths SameDiff sd = SameDiff.load(new File("model.sdz"), false); SameDiff sd = SameDiff.load(new File("model.sdnb"), false); SameDiff sd = SameDiff.load(new File("model.fb"), false); // legacy SameDiff sd = SameDiff.load(new File("large_model"), false); // sharded base name ``` ### Metadata Both SDNB and SDZ support an extensible key-value metadata map (similar to GGUF). Metadata can be added after the model is saved without reserializing the weight tensors. Standardized keys include: | Key | Meaning | | --------------------- | ----------------------------------- | | `model.name` | Human-readable model name | | `model.version` | Semantic version string | | `training.epochs` | Number of training epochs completed | | `training.dataset` | Dataset used for training | | `nd4j.format.version` | Format version (set automatically) | ### When to Use Each Format | Scenario | Recommended Format | | --------------------------------------- | -------------------------------------------- | | Training checkpoints (fast save/load) | SDNB | | Resuming training | SDNB with `saveUpdaterState=true` | | Deployment / single-file distribution | SDZ | | Models larger than 2 GB | SDZ (auto-shards internally) or sharded SDNB | | Inspecting contents with standard tools | SDZ (`unzip -l model.sdz`) | | Migrating an existing `.fb` model | Load `.fb`, save as `.sdz` | ### Backward Compatibility Existing `.fb` files continue to load without any changes — the auto-detection path checks for the legacy format first. No migration is required for models already in production. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://deeplearning4j.konduit.ai/en-1.0.0-rewrite/nd4j/overview-2/serialization.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.