> For the complete documentation index, see [llms.txt](https://deeplearning4j.konduit.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://deeplearning4j.konduit.ai/en-1.0.0-rewrite/configuration/graalvm.md).

# GraalVM Native Image

### Overview

GraalVM Native Image compiles a Java application ahead-of-time into a platform-native executable. The result starts in milliseconds rather than seconds, consumes less memory at idle, and ships as a single self-contained binary with no JVM installation required.

ND4J's use of reflection (the SameDiff op registry and the `DifferentialFunction` hierarchy) and JNI (JavaCPP native bindings to libnd4j, OpenBLAS, MKL, oneDNN, and cuDNN) requires explicit metadata for Native Image to compile correctly. Starting with the 1.0.0-rewrite, ND4J ships this metadata in `META-INF/native-image/` directories inside each JAR so that no hand-written configuration is needed for the common case.

This page covers what the metadata provides, how to build a native executable, and how to extend the configuration for custom ops or application-level reflection.

### Use Cases

GraalVM Native Image is well-suited for:

* **Serverless inference endpoints** — cold-start latency is a primary cost driver for functions-as-a-service. Native Image drops startup from 2–10 seconds (JVM) to under 100 milliseconds.
* **Edge and IoT deployment** — single-binary distribution without a JVM runtime; smaller container images.
* **Command-line tools** — fast response time for model evaluation scripts, data preprocessing pipelines, and CLI utilities built on ND4J.
* **Sidecar services** — lightweight inference pods in a Kubernetes cluster where JVM memory overhead per container is unacceptable.

GraalVM Native Image is not recommended for training workloads or for applications that rely heavily on dynamic class loading at runtime (e.g., Keras model import from arbitrary Python-generated protobuf files with unknown op sets).

### Bundled `META-INF/native-image/` Configuration

ND4J ships Native Image configuration in three modules:

#### nd4j-api

* **`reflect-config.json`** — Registers the full SameDiff and op class hierarchy for reflective access: `DifferentialFunction`, `SameDiff`, and all classes in the SD op namespaces (activations, linalg, math, nn, etc.).
* **`native-image.properties`** — Defers initialization of `org.nd4j.**` and `org.bytedeco.**` to runtime (these classes probe native libraries and call `Unsafe` during their static initializers, which is unsafe at build time). Eagerly initializes `org.nd4j.shade.protobuf.*` at build time, which is safe and avoids protobuf descriptor rebuilding on every startup.

#### nd4j-native (CPU backend)

* **`reflect-config.json`** — CPU backend classes: `NDArray`, `CpuMemoryManager`, `Nd4jCpu`, and related helpers.
* **`jni-config.json`** — JNI method signatures for `NativeOps` and `NativeOpsHolder`.
* **`resource-config.json`** — Bundles native shared libraries for all six supported platforms so they are included in the native executable and extracted at runtime by JavaCPP's Loader:
  * `linux-x86_64`
  * `linux-aarch64`
  * `windows-x86_64`
  * `macosx-x86_64`
  * `macosx-arm64`
  * `android-arm64`
* **`native-image.properties`** — Adds `-H:+JNI` and defers the full JavaCPP native chain (`javacpp.Loader`, `Pointer`, OpenBLAS, MKL, oneDNN) to runtime.

#### nd4j-cuda (CUDA backend)

Parallel configuration for CUDA-specific classes: `CudaMemoryManager`, `CudaAffinityManager`, `Nd4jCuda`, cuDNN helpers, and associated JNI bindings.

#### Initialization Strategy Summary

| Package / Class                | Init Time  | Reason                                             |
| ------------------------------ | ---------- | -------------------------------------------------- |
| `org.nd4j.shade.protobuf.*`    | Build time | Safe; no native probes                             |
| `org.nd4j.**`                  | Runtime    | Calls `Unsafe`, probes native libs                 |
| `org.bytedeco.**`              | Runtime    | JavaCPP `Loader` runs native extraction            |
| `org.nd4j.nativeblas.Nd4jCpu`  | Runtime    | JNI class; must load after native lib is extracted |
| `org.nd4j.nativeblas.Nd4jCuda` | Runtime    | JNI class; same reason                             |

### Prerequisites

1. **GraalVM JDK 21+** with the Native Image component installed.

   ```bash
   # Install GraalVM via SDKMAN (recommended)
   sdk install java 21.0.3-graal
   sdk use java 21.0.3-graal

   # Install native-image tool
   gu install native-image

   # Verify
   native-image --version
   ```
2. **System build tools** — GCC/Clang, `zlib`, `libstdc++`. On Ubuntu/Debian:

   ```bash
   sudo apt-get install build-essential zlib1g-dev
   ```
3. **For CUDA native images** — CUDA toolkit must be installed on the build machine, and the target machine must have the same CUDA version. Native Image executables embed the JavaCPP-extracted shared library path at build time.

### Maven Setup

Add the GraalVM Native Image Maven plugin alongside your normal ND4J dependencies:

```xml
<properties>
    <dl4j.version>1.0.0-rewrite</dl4j.version>
    <graalvm.version>0.10.2</graalvm.version>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-bom</artifactId>
            <version>${dl4j.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- Core DL4J (optional — omit if using ND4J / SameDiff only) -->
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-core</artifactId>
    </dependency>

    <!-- CPU backend — use -lite classifier for smallest binary -->
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-native-platform</artifactId>
    </dependency>

    <!-- Logging (required; logback works with Native Image) -->
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.4.14</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.graalvm.buildtools</groupId>
            <artifactId>native-maven-plugin</artifactId>
            <version>${graalvm.version}</version>
            <extensions>true</extensions>
            <configuration>
                <imageName>my-nd4j-app</imageName>
                <mainClass>com.example.MyInferenceApp</mainClass>
                <buildArgs>
                    <!-- Initialize nd4j and bytedeco at runtime, not build time -->
                    <buildArg>--initialize-at-run-time=org.nd4j,org.bytedeco</buildArg>
                    <!-- Allow JNI (required for JavaCPP native bindings) -->
                    <buildArg>-H:+JNI</buildArg>
                    <!-- Report any missing reflection config as build-time errors -->
                    <buildArg>--no-fallback</buildArg>
                    <!-- Optional: reduce output size on Linux -->
                    <buildArg>-march=x86-64-v3</buildArg>
                </buildArgs>
            </configuration>
            <executions>
                <execution>
                    <id>build-native</id>
                    <goals>
                        <goal>compile-no-fork</goal>
                    </goals>
                    <phase>package</phase>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
```

### Build Commands

```bash
# Standard Maven build + native image (runs during package phase)
mvn -Pnative package

# Or invoke the plugin goal directly
mvn native:compile-no-fork

# The resulting executable is at:
./target/my-nd4j-app

# Run it
./target/my-nd4j-app
```

Native Image compilation is slow (typically 2–5 minutes for a full DL4J application). For development, keep using the standard JVM; switch to native only for CI release builds.

### Building a CPU-Only Inference Executable

The smallest possible native image uses the `-lite` classifier (limited type set) and does not include cuDNN or CUDA:

```xml
<!-- Swap nd4j-native-platform for the lite single-platform classifier -->
<dependency>
    <groupId>org.nd4j</groupId>
    <artifactId>nd4j-native</artifactId>
    <version>${dl4j.version}</version>
    <classifier>linux-x86_64-lite</classifier>
</dependency>
```

```bash
mvn native:compile-no-fork
ls -lh target/my-nd4j-app
# Typical size: 35–60 MB including embedded libnd4jcpu.so
```

### Building a CUDA-Enabled Executable

CUDA native images embed the CUDA native library. Build and target machines must have the same CUDA version.

```xml
<!-- Replace nd4j-native-platform with CUDA backend -->
<dependency>
    <groupId>org.nd4j</groupId>
    <artifactId>nd4j-cuda-12.9-platform</artifactId>
    <version>${dl4j.version}</version>
</dependency>
```

```bash
# Build on a machine with CUDA 12.9 installed
mvn native:compile-no-fork -Dnd4j.cuda.version=12.9

# The executable embeds libnd4jcuda.so; run on a machine with GPU + CUDA 12.9 runtime
./target/my-nd4j-app
```

### Adding Reflection Config for Custom Classes

If your application uses `@JsonSubTypes`, `@EqualsAndHashCode`, or any reflective instantiation of ND4J subclasses not in the standard op registry, you need to add them to the reflection configuration.

Create `src/main/resources/META-INF/native-image/com.example/my-app/reflect-config.json`:

```json
[
  {
    "name": "com.example.MyCustomOp",
    "allDeclaredConstructors": true,
    "allPublicMethods": true
  },
  {
    "name": "com.example.MyCustomLayer",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true
  }
]
```

GraalVM's agent-based metadata collection is the easiest way to discover missing registrations. Run your application with the tracing agent on the JVM first, then use the generated config as a starting point:

```bash
# Run with tracing agent to discover reflection/JNI usage
java -agentlib:native-image-agent=config-output-dir=src/main/resources/META-INF/native-image/com.example/my-app \
     -jar target/my-nd4j-app.jar

# Then build native image — it picks up the generated config automatically
mvn native:compile-no-fork
```

### Startup Time Comparison

| Mode                     | Startup Time (typical) | Notes                                       |
| ------------------------ | ---------------------- | ------------------------------------------- |
| Standard JVM (no warmup) | 2–8 seconds            | Includes class loading, JIT warmup          |
| GraalVM JVM mode         | 1–4 seconds            | GraalVM JIT, no AOT compilation             |
| GraalVM Native Image     | 30–150 ms              | AOT compiled; no JVM startup, no JIT warmup |

The native image startup advantage is most significant for short-lived inference calls (REST handler, Lambda function, CLI tool). For long-running server processes, the JVM's JIT compiler eventually produces comparable or better throughput.

### Known Limitations

* **Per-platform builds** — a native executable compiled on Linux x86-64 does not run on macOS or ARM. You need separate builds per target platform (or use a CI matrix).
* **Reflection completeness** — every class accessed reflectively at runtime must be registered at build time. New custom ops, Jackson-serialized model configs, or dynamic SPI registrations require updating `reflect-config.json`.
* **No JIT after startup** — once compiled, the native image cannot JIT-optimize hot paths discovered at runtime. Peak throughput for sustained large-batch workloads may be lower than with the JVM.
* **CUDA version pinning** — the CUDA runtime version is baked in at build time. Upgrading CUDA on the deployment host requires rebuilding the native image.
* **Training not recommended** — GraalVM Native Image does not support `java.lang.instrument` or class redefinition, which some DL4J training listeners use. Native Image is best suited for inference.

### Java Module System (JPMS) Compatibility

Starting with the 1.0.0-rewrite, ND4J ships `module-info.java` descriptors in its core modules. This lets you use ND4J inside a fully modularized Java application (JPMS / Project Jigsaw) without `--add-opens` hacks, and it ensures compatibility with Java 17 and later LTS releases where the module system is enforced more strictly.

#### Shipped Module Descriptors

Three ND4J modules carry explicit `module-info.java` descriptors:

| Maven Artifact               | JPMS Module Name  |
| ---------------------------- | ----------------- |
| `nd4j-api`                   | `org.nd4j.api`    |
| `nd4j-native` (CPU backend)  | `org.nd4j.native` |
| `nd4j-cuda-*` (CUDA backend) | `org.nd4j.cuda`   |

The descriptors are generated and embedded at build time using the [moditect Maven plugin](https://github.com/moditect/moditect). Moditect is invoked selectively — it does not run in every build by default, only during the release publication workflow — so the metadata is present in all published JARs on Maven Central without affecting incremental local builds.

Each backend JAR (nd4j-native, nd4j-cuda) has a unique module name per classifier. This avoids the `duplicate module name` error that would otherwise occur when both jars appear on the module path simultaneously.

#### Java 17 Compatibility Notes

* The module descriptors allow ND4J to be placed on the **module path** (`--module-path`) rather than the traditional class path. Both arrangements continue to work; the module path is optional.
* Packages that were previously split across jars have been reorganized so each package lives in exactly one module. Some internal classes (primarily preset names and internal registry helpers) were moved as part of this change. Public APIs are unaffected.
* Backwards compatibility with Java 8 is preserved: the moditect approach injects `module-info.class` files into the JAR at the `META-INF/versions/9/` multi-release path, so the same JAR runs on Java 8 without the descriptor causing any errors.

#### Using ND4J in a Modularized Application

Add `requires` declarations for the ND4J modules you use. The minimum for CPU inference:

```java
// src/main/java/module-info.java
module com.example.myapp {
    requires org.nd4j.api;
    requires org.nd4j.native;   // CPU backend

    // Optional: if using the CUDA backend instead of or in addition to CPU
    // requires org.nd4j.cuda;

    // Export your own packages if needed
    exports com.example.myapp.inference;
}
```

For CUDA-only inference, swap `org.nd4j.native` for `org.nd4j.cuda`:

```java
module com.example.myapp {
    requires org.nd4j.api;
    requires org.nd4j.cuda;

    exports com.example.myapp;
}
```

#### Maven Module-Path Setup

By default, Maven puts all dependencies on the class path. To place ND4J on the module path, configure the `maven-compiler-plugin` to treat the ND4J modules explicitly:

```xml
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.13.0</version>
    <configuration>
        <release>17</release>
    </configuration>
</plugin>
```

If your `module-info.java` is present in `src/main/java/`, the compiler resolves `requires` declarations automatically from the module path. No additional plugin configuration is needed beyond the standard `<release>17</release>` setting.

#### Moditect Plugin (Advanced)

If you are building a downstream library on top of ND4J and need to add module metadata to jars that do not yet have it, you can invoke the moditect plugin in your own build:

```xml
<plugin>
    <groupId>org.moditect</groupId>
    <artifactId>moditect-maven-plugin</artifactId>
    <version>1.2.2.Final</version>
    <executions>
        <execution>
            <id>add-module-infos</id>
            <phase>package</phase>
            <goals>
                <goal>add-module-info</goal>
            </goals>
            <configuration>
                <overwriteExistingFiles>true</overwriteExistingFiles>
                <modules>
                    <module>
                        <artifact>
                            <groupId>com.example</groupId>
                            <artifactId>my-nd4j-extension</artifactId>
                        </artifact>
                        <moduleInfoSource>
                            module com.example.nd4jextension {
                                requires org.nd4j.api;
                                exports com.example.nd4jextension;
                            }
                        </moduleInfoSource>
                    </module>
                </modules>
            </configuration>
        </execution>
    </executions>
</plugin>
```

The moditect `add-module-info` goal injects a `module-info.class` into an existing JAR. This is the same mechanism ND4J itself uses internally.

#### JPMS and GraalVM Native Image

When building a GraalVM Native Image from a modularized application, the native-image tool reads `module-info.class` descriptors from the module path. The ND4J module descriptors are compatible with the native-image build process. Combine the module-path setup above with the `native-maven-plugin` configuration described in the [Maven Setup](#maven-setup) section; no additional flags are required.

#### Known Limitations

* **OSGi**: The JPMS module names follow the `org.nd4j.*` namespace but OSGi bundle headers (`Bundle-SymbolicName`, `Export-Package`) are separate metadata not currently shipped. OSGi integration requires a separate bundling step (e.g., bnd tool).
* **Renamed internal classes**: If you were using internal classes that moved as part of the split-package resolution (primarily in preset name registries), update your imports. All public APIs in `org.nd4j.linalg.*` and `org.deeplearning4j.*` are unaffected.

***

### See Also

* [Maven Setup](https://github.com/KonduitAI/deeplearning4j-docs/blob/en-1.0.0-rewrite/docs/m2.1/config/maven/README.md) — dependency management, BOM, classifier variants
* [Hardware Backends](https://github.com/KonduitAI/deeplearning4j-docs/blob/en-1.0.0-rewrite/docs/m2.1/nd4j/backends/hardware-backends/README.md) — backend selection and configuration
* [Snapshots](https://github.com/KonduitAI/deeplearning4j-docs/blob/en-1.0.0-rewrite/docs/m2.1/config/snapshots/README.md) — using nightly builds with Native Image


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://deeplearning4j.konduit.ai/en-1.0.0-rewrite/configuration/graalvm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.