> For the complete documentation index, see [llms.txt](https://deeplearning4j.konduit.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://deeplearning4j.konduit.ai/en-1.0.0-rewrite/configuration/workspaces.md). # Workspaces ### What Are Workspaces? ND4J workspaces are a memory management model designed for cyclic workloads like neural network training and inference. Instead of allocating and releasing off-heap memory for each `INDArray` independently (which triggers garbage collection), workspaces maintain a fixed-size memory region that is reused across iterations. At the end of each workspace cycle, all `INDArray` objects created within the workspace have their memory invalidated and made available for reuse in the next cycle. No deallocation or GC pressure occurs. This makes workspace-based memory management significantly faster and more predictable than relying on the JVM GC for off-heap memory reclamation. DL4J enables workspaces by default for all `MultiLayerNetwork` and `ComputationGraph` instances from version 1.0.0-alpha onwards. For most users, no explicit configuration is needed. ### WorkspaceMode Enum `WorkspaceMode` controls how workspaces are used during training and inference. #### ENABLED (recommended) ```java new NeuralNetConfiguration.Builder() .trainingWorkspaceMode(WorkspaceMode.ENABLED) .inferenceWorkspaceMode(WorkspaceMode.ENABLED) // ... ``` `ENABLED` is the default and the recommended setting. All intermediate activations, gradients, and updater state during a training or inference pass are allocated within a workspace and reused across iterations. This results in: * Minimal GC pressure during training * Consistent per-iteration memory usage after the first few warmup iterations * Better performance than `NONE` #### NONE ```java new NeuralNetConfiguration.Builder() .trainingWorkspaceMode(WorkspaceMode.NONE) .inferenceWorkspaceMode(WorkspaceMode.NONE) // ... ``` `NONE` disables workspace-based memory management entirely. Every `INDArray` is allocated independently and freed by the JVM GC when no longer referenced. This is slower and less memory-efficient but is occasionally useful for debugging. When to use `NONE`: * Debugging a `ND4JIllegalStateException` related to workspace pointer leaks * Isolating whether a bug is workspace-related or not * Custom layers or data pipelines that are not workspace-aware #### SEPARATE and SINGLE (deprecated) Prior to 1.0.0-alpha, DL4J used `SEPARATE` and `SINGLE` modes: * `SEPARATE` — used separate workspaces for different stages of the forward/backward pass. Slightly slower, less memory. * `SINGLE` — used one workspace for all stages. Slightly faster, more memory. Both are deprecated and functionally replaced by `ENABLED`. Do not use them in new code. ### Checking and Changing Workspace Configuration #### MultiLayerNetwork ```java // Check current workspace mode System.out.println("Training workspace: " + net.getLayerWiseConfigurations().getTrainingWorkspaceMode()); System.out.println("Inference workspace: " + net.getLayerWiseConfigurations().getInferenceWorkspaceMode()); // Change workspace mode on an existing network net.getLayerWiseConfigurations().setTrainingWorkspaceMode(WorkspaceMode.ENABLED); net.getLayerWiseConfigurations().setInferenceWorkspaceMode(WorkspaceMode.ENABLED); ``` #### ComputationGraph ```java System.out.println("Training workspace: " + cg.getConfiguration().getTrainingWorkspaceMode()); System.out.println("Inference workspace: " + cg.getConfiguration().getInferenceWorkspaceMode()); cg.getConfiguration().setTrainingWorkspaceMode(WorkspaceMode.ENABLED); cg.getConfiguration().setInferenceWorkspaceMode(WorkspaceMode.ENABLED); ``` ### Garbage Collector Interaction When workspaces are enabled, ND4J's periodic `System.gc()` calls become less important because off-heap memory is managed by the workspace rather than by GC-driven WeakReference cleanup. It is beneficial to reduce or disable them: ```java // Reduce GC call frequency to once every 10 seconds Nd4j.getMemoryManager().setAutoGcWindow(10000); // Or disable periodic GC entirely (safe with ENABLED workspaces) Nd4j.getMemoryManager().togglePeriodicGc(false); ``` Place these before `model.fit(...)` or before the inference loop. ### ParallelWrapper and Multi-GPU `ParallelWrapper` uses one workspace per training thread: ```java ParallelWrapper wrapper = new ParallelWrapper.Builder(model) .prefetchBuffer(8) .workers(4) // one per GPU .averagingFrequency(3) .workspaceMode(WorkspaceMode.ENABLED) .build(); ``` ### Moving Arrays Out of a Workspace Arrays created inside a workspace are valid only within the workspace's scope. If you need to retain an array after the workspace cycle ends, call `detach()`: ```java INDArray predictions = model.output(input); // 'predictions' is detached from the workspace by model.output() automatically. // For custom code inside a workspace scope: try (MemoryWorkspace ws = Nd4j.getWorkspaceManager().getAndActivateWorkspace("MY_WS")) { INDArray temp = Nd4j.create(1000); INDArray persistent = temp.detach(); // copy out of workspace into GC-managed memory } // temp is now invalidated; persistent is still valid ``` ### Async Iterators and Workspaces `AsyncDataSetIterator` and `AsyncMultiDataSetIterator` use a cyclic workspace internally to prefetch minibatches. This means the `DataSet` objects returned by `next()` are workspace-managed and will be overwritten on subsequent `next()` calls. **Important:** Consume each `DataSet` from an async iterator before calling `next()` again. Do not store `DataSet` references from an async iterator without calling `detach()` on its arrays first. If you need to disable async prefetch for debugging: ```java DataSetIterator wrapped = new AsyncShieldDataSetIterator(yourIterator); ``` ### Destroying Workspaces In low-memory situations or between training runs, you can release all workspaces for the current thread: ```java Nd4j.getWorkspaceManager().destroyAllWorkspacesForCurrentThread(); ``` To release workspaces in a specific thread from another thread, call the same method from within that thread. ### Debugging Workspace Issues If you see an error like: ``` org.nd4j.linalg.exception.ND4JIllegalStateException: Op [set] Y argument uses leaked workspace pointer from workspace [LOOP_EXTERNAL] For more details, see the ND4J User Guide: nd4j.org/userguide#workspaces-panic ``` This means an array allocated inside a workspace is being used outside its scope. Common causes: 1. A custom layer is storing an activation array reference across iterations without calling `detach()`. 2. A custom data iterator returns arrays backed by workspace memory that is later invalidated. 3. An array is passed to code that stores it, such as a listener or evaluator that retains it beyond the iteration. **Debugging approach:** 1. Temporarily switch to `WorkspaceMode.NONE` to see if the error disappears. If it does, the issue is a workspace scope violation. 2. Audit custom layer code: all returned arrays from `activate()` should be in `ArrayType.ACTIVATIONS`, and arrays from `backpropGradient()` should be in `ArrayType.ACTIVATION_GRAD`, using `LayerWorkspaceMgr`. 3. Use `LayerWorkspaceMgr.noWorkspaces()` when doing forward passes outside of a `MultiLayerNetwork`/`ComputationGraph` context. ### Evaluation Without Creating New Arrays The preferred way to evaluate a model during training avoids creating extra `INDArray` allocations: ```java Evaluation eval = new Evaluation(numClasses); ROC roc = new ROC(100); // Runs a single pass over the iterator, updating all IEvaluation instances model.doEvaluation(testIterator, eval, roc); System.out.println(eval.stats()); ``` This is more efficient than calling `model.output()` in a loop and storing predictions. ### Custom Workspace Usage For custom cyclic workloads outside of DL4J's training loop: ```java WorkspaceConfiguration config = WorkspaceConfiguration.builder() .initialSize(100 * 1024 * 1024L) // 100 MB initial size .overallocationLimit(0.1) // Allow 10% overallocation .policyAllocation(AllocationPolicy.OVERALLOCATE) .policyLearning(LearningPolicy.FIRST_LOOP) .policySpill(SpillPolicy.REALLOCATE) .policyReset(ResetPolicy.BLOCK_LEFT) .build(); for (int i = 0; i < numIterations; i++) { try (MemoryWorkspace ws = Nd4j.getWorkspaceManager() .getAndActivateWorkspace(config, "MY_CYCLIC_WS")) { // All Nd4j.create() calls here are allocated in the workspace INDArray a = Nd4j.create(1000); INDArray b = Nd4j.create(1000); INDArray result = a.add(b); // result is valid until end of this try block } // workspace memory is reset for next iteration; no GC needed } ``` ### Related Pages * [Memory Configuration](https://github.com/KonduitAI/deeplearning4j-docs/blob/en-1.0.0-rewrite/docs/m2.1/config/memory/README.md) — JVM and off-heap memory flags * [Performance Debugging](https://github.com/KonduitAI/deeplearning4j-docs/blob/en-1.0.0-rewrite/docs/m2.1/config/performance-debugging/README.md) — diagnosing GC and memory issues * [GPU and CPU Setup](https://github.com/KonduitAI/deeplearning4j-docs/blob/en-1.0.0-rewrite/docs/m2.1/config/gpu-cpu/README.md) — backend setup --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://deeplearning4j.konduit.ai/en-1.0.0-rewrite/configuration/workspaces.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.