GraalVM Native Image

Build ND4J and DL4J applications as GraalVM native executables for millisecond startup — serverless, edge, and single-binary deployment

Overview

GraalVM Native Image compiles a Java application ahead-of-time into a platform-native executable. The result starts in milliseconds rather than seconds, consumes less memory at idle, and ships as a single self-contained binary with no JVM installation required.

ND4J's use of reflection (the SameDiff op registry and the DifferentialFunction hierarchy) and JNI (JavaCPP native bindings to libnd4j, OpenBLAS, MKL, oneDNN, and cuDNN) requires explicit metadata for Native Image to compile correctly. Starting with the 1.0.0-rewrite, ND4J ships this metadata in META-INF/native-image/ directories inside each JAR so that no hand-written configuration is needed for the common case.

This page covers what the metadata provides, how to build a native executable, and how to extend the configuration for custom ops or application-level reflection.

Use Cases

GraalVM Native Image is well-suited for:

• Serverless inference endpoints — cold-start latency is a primary cost driver for functions-as-a-service. Native Image drops startup from 2–10 seconds (JVM) to under 100 milliseconds.

• Edge and IoT deployment — single-binary distribution without a JVM runtime; smaller container images.

• Command-line tools — fast response time for model evaluation scripts, data preprocessing pipelines, and CLI utilities built on ND4J.

• Sidecar services — lightweight inference pods in a Kubernetes cluster where JVM memory overhead per container is unacceptable.

GraalVM Native Image is not recommended for training workloads or for applications that rely heavily on dynamic class loading at runtime (e.g., Keras model import from arbitrary Python-generated protobuf files with unknown op sets).

Bundled META-INF/native-image/ Configuration

ND4J ships Native Image configuration in three modules:

nd4j-api

• reflect-config.json — Registers the full SameDiff and op class hierarchy for reflective access: DifferentialFunction, SameDiff, and all classes in the SD op namespaces (activations, linalg, math, nn, etc.).

• native-image.properties — Defers initialization of org.nd4j.** and org.bytedeco.** to runtime (these classes probe native libraries and call Unsafe during their static initializers, which is unsafe at build time). Eagerly initializes org.nd4j.shade.protobuf.* at build time, which is safe and avoids protobuf descriptor rebuilding on every startup.

nd4j-native (CPU backend)

• reflect-config.json — CPU backend classes: NDArray, CpuMemoryManager, Nd4jCpu, and related helpers.

• jni-config.json — JNI method signatures for NativeOps and NativeOpsHolder.

• resource-config.json — Bundles native shared libraries for all six supported platforms so they are included in the native executable and extracted at runtime by JavaCPP's Loader:

  • linux-x86_64

  • linux-aarch64

  • windows-x86_64

  • macosx-x86_64

  • macosx-arm64

  • android-arm64

• native-image.properties — Adds -H:+JNI and defers the full JavaCPP native chain (javacpp.Loader, Pointer, OpenBLAS, MKL, oneDNN) to runtime.

nd4j-cuda (CUDA backend)

Parallel configuration for CUDA-specific classes: CudaMemoryManager, CudaAffinityManager, Nd4jCuda, cuDNN helpers, and associated JNI bindings.

Initialization Strategy Summary

Package / Class	Init Time	Reason
org.nd4j.shade.protobuf.*	Build time	Safe; no native probes
org.nd4j.**	Runtime	Calls Unsafe, probes native libs
org.bytedeco.**	Runtime	JavaCPP Loader runs native extraction
org.nd4j.nativeblas.Nd4jCpu	Runtime	JNI class; must load after native lib is extracted
org.nd4j.nativeblas.Nd4jCuda	Runtime	JNI class; same reason

Prerequisites

1. GraalVM JDK 21+ with the Native Image component installed.

   # Install GraalVM via SDKMAN (recommended)
   sdk install java 21.0.3-graal
   sdk use java 21.0.3-graal
   
   # Install native-image tool
   gu install native-image
   
   # Verify
   native-image --version

2. System build tools — GCC/Clang, zlib, libstdc++. On Ubuntu/Debian:

   sudo apt-get install build-essential zlib1g-dev

3. For CUDA native images — CUDA toolkit must be installed on the build machine, and the target machine must have the same CUDA version. Native Image executables embed the JavaCPP-extracted shared library path at build time.

Maven Setup

Add the GraalVM Native Image Maven plugin alongside your normal ND4J dependencies:

<properties>
    <dl4j.version>1.0.0-rewrite</dl4j.version>
    <graalvm.version>0.10.2</graalvm.version>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-bom</artifactId>
            <version>${dl4j.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- Core DL4J (optional — omit if using ND4J / SameDiff only) -->
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-core</artifactId>
    </dependency>

    <!-- CPU backend — use -lite classifier for smallest binary -->
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-native-platform</artifactId>
    </dependency>

    <!-- Logging (required; logback works with Native Image) -->
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.4.14</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.graalvm.buildtools</groupId>
            <artifactId>native-maven-plugin</artifactId>
            <version>${graalvm.version}</version>
            <extensions>true</extensions>
            <configuration>
                <imageName>my-nd4j-app</imageName>
                <mainClass>com.example.MyInferenceApp</mainClass>
                <buildArgs>
                    <!-- Initialize nd4j and bytedeco at runtime, not build time -->
                    <buildArg>--initialize-at-run-time=org.nd4j,org.bytedeco</buildArg>
                    <!-- Allow JNI (required for JavaCPP native bindings) -->
                    <buildArg>-H:+JNI</buildArg>
                    <!-- Report any missing reflection config as build-time errors -->
                    <buildArg>--no-fallback</buildArg>
                    <!-- Optional: reduce output size on Linux -->
                    <buildArg>-march=x86-64-v3</buildArg>
                </buildArgs>
            </configuration>
            <executions>
                <execution>
                    <id>build-native</id>
                    <goals>
                        <goal>compile-no-fork</goal>
                    </goals>
                    <phase>package</phase>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Build Commands

# Standard Maven build + native image (runs during package phase)
mvn -Pnative package

# Or invoke the plugin goal directly
mvn native:compile-no-fork

# The resulting executable is at:
./target/my-nd4j-app

# Run it
./target/my-nd4j-app

Native Image compilation is slow (typically 2–5 minutes for a full DL4J application). For development, keep using the standard JVM; switch to native only for CI release builds.

Building a CPU-Only Inference Executable

The smallest possible native image uses the -lite classifier (limited type set) and does not include cuDNN or CUDA:

<!-- Swap nd4j-native-platform for the lite single-platform classifier -->
<dependency>
    <groupId>org.nd4j</groupId>
    <artifactId>nd4j-native</artifactId>
    <version>${dl4j.version}</version>
    <classifier>linux-x86_64-lite</classifier>
</dependency>

mvn native:compile-no-fork
ls -lh target/my-nd4j-app
# Typical size: 35–60 MB including embedded libnd4jcpu.so

Building a CUDA-Enabled Executable

CUDA native images embed the CUDA native library. Build and target machines must have the same CUDA version.

<!-- Replace nd4j-native-platform with CUDA backend -->
<dependency>
    <groupId>org.nd4j</groupId>
    <artifactId>nd4j-cuda-12.9-platform</artifactId>
    <version>${dl4j.version}</version>
</dependency>

# Build on a machine with CUDA 12.9 installed
mvn native:compile-no-fork -Dnd4j.cuda.version=12.9

# The executable embeds libnd4jcuda.so; run on a machine with GPU + CUDA 12.9 runtime
./target/my-nd4j-app

Adding Reflection Config for Custom Classes

If your application uses @JsonSubTypes, @EqualsAndHashCode, or any reflective instantiation of ND4J subclasses not in the standard op registry, you need to add them to the reflection configuration.

Create src/main/resources/META-INF/native-image/com.example/my-app/reflect-config.json:

[
  {
    "name": "com.example.MyCustomOp",
    "allDeclaredConstructors": true,
    "allPublicMethods": true
  },
  {
    "name": "com.example.MyCustomLayer",
    "allDeclaredFields": true,
    "allDeclaredConstructors": true
  }
]

GraalVM's agent-based metadata collection is the easiest way to discover missing registrations. Run your application with the tracing agent on the JVM first, then use the generated config as a starting point:

# Run with tracing agent to discover reflection/JNI usage
java -agentlib:native-image-agent=config-output-dir=src/main/resources/META-INF/native-image/com.example/my-app \
     -jar target/my-nd4j-app.jar

# Then build native image — it picks up the generated config automatically
mvn native:compile-no-fork

Startup Time Comparison

Mode	Startup Time (typical)	Notes
Standard JVM (no warmup)	2–8 seconds	Includes class loading, JIT warmup
GraalVM JVM mode	1–4 seconds	GraalVM JIT, no AOT compilation
GraalVM Native Image	30–150 ms	AOT compiled; no JVM startup, no JIT warmup

The native image startup advantage is most significant for short-lived inference calls (REST handler, Lambda function, CLI tool). For long-running server processes, the JVM's JIT compiler eventually produces comparable or better throughput.

Known Limitations

• Per-platform builds — a native executable compiled on Linux x86-64 does not run on macOS or ARM. You need separate builds per target platform (or use a CI matrix).

• Reflection completeness — every class accessed reflectively at runtime must be registered at build time. New custom ops, Jackson-serialized model configs, or dynamic SPI registrations require updating reflect-config.json.

• No JIT after startup — once compiled, the native image cannot JIT-optimize hot paths discovered at runtime. Peak throughput for sustained large-batch workloads may be lower than with the JVM.

• CUDA version pinning — the CUDA runtime version is baked in at build time. Upgrading CUDA on the deployment host requires rebuilding the native image.

• Training not recommended — GraalVM Native Image does not support java.lang.instrument or class redefinition, which some DL4J training listeners use. Native Image is best suited for inference.

Java Module System (JPMS) Compatibility

Starting with the 1.0.0-rewrite, ND4J ships module-info.java descriptors in its core modules. This lets you use ND4J inside a fully modularized Java application (JPMS / Project Jigsaw) without --add-opens hacks, and it ensures compatibility with Java 17 and later LTS releases where the module system is enforced more strictly.

Shipped Module Descriptors

Three ND4J modules carry explicit module-info.java descriptors:

Maven Artifact	JPMS Module Name
nd4j-api	org.nd4j.api
nd4j-native (CPU backend)	org.nd4j.native
nd4j-cuda-* (CUDA backend)	org.nd4j.cuda

The descriptors are generated and embedded at build time using the moditect Maven plugin. Moditect is invoked selectively — it does not run in every build by default, only during the release publication workflow — so the metadata is present in all published JARs on Maven Central without affecting incremental local builds.

Each backend JAR (nd4j-native, nd4j-cuda) has a unique module name per classifier. This avoids the duplicate module name error that would otherwise occur when both jars appear on the module path simultaneously.

Java 17 Compatibility Notes

• The module descriptors allow ND4J to be placed on the module path (--module-path) rather than the traditional class path. Both arrangements continue to work; the module path is optional.

• Packages that were previously split across jars have been reorganized so each package lives in exactly one module. Some internal classes (primarily preset names and internal registry helpers) were moved as part of this change. Public APIs are unaffected.

• Backwards compatibility with Java 8 is preserved: the moditect approach injects module-info.class files into the JAR at the META-INF/versions/9/ multi-release path, so the same JAR runs on Java 8 without the descriptor causing any errors.

Using ND4J in a Modularized Application

Add requires declarations for the ND4J modules you use. The minimum for CPU inference:

// src/main/java/module-info.java
module com.example.myapp {
    requires org.nd4j.api;
    requires org.nd4j.native;   // CPU backend

    // Optional: if using the CUDA backend instead of or in addition to CPU
    // requires org.nd4j.cuda;

    // Export your own packages if needed
    exports com.example.myapp.inference;
}

For CUDA-only inference, swap org.nd4j.native for org.nd4j.cuda:

module com.example.myapp {
    requires org.nd4j.api;
    requires org.nd4j.cuda;

    exports com.example.myapp;
}

Maven Module-Path Setup

By default, Maven puts all dependencies on the class path. To place ND4J on the module path, configure the maven-compiler-plugin to treat the ND4J modules explicitly:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.13.0</version>
    <configuration>
        <release>17</release>
    </configuration>
</plugin>

If your module-info.java is present in src/main/java/, the compiler resolves requires declarations automatically from the module path. No additional plugin configuration is needed beyond the standard <release>17</release> setting.

Moditect Plugin (Advanced)

If you are building a downstream library on top of ND4J and need to add module metadata to jars that do not yet have it, you can invoke the moditect plugin in your own build:

<plugin>
    <groupId>org.moditect</groupId>
    <artifactId>moditect-maven-plugin</artifactId>
    <version>1.2.2.Final</version>
    <executions>
        <execution>
            <id>add-module-infos</id>
            <phase>package</phase>
            <goals>
                <goal>add-module-info</goal>
            </goals>
            <configuration>
                <overwriteExistingFiles>true</overwriteExistingFiles>
                <modules>
                    <module>
                        <artifact>
                            <groupId>com.example</groupId>
                            <artifactId>my-nd4j-extension</artifactId>
                        </artifact>
                        <moduleInfoSource>
                            module com.example.nd4jextension {
                                requires org.nd4j.api;
                                exports com.example.nd4jextension;
                            }
                        </moduleInfoSource>
                    </module>
                </modules>
            </configuration>
        </execution>
    </executions>
</plugin>

The moditect add-module-info goal injects a module-info.class into an existing JAR. This is the same mechanism ND4J itself uses internally.

JPMS and GraalVM Native Image

When building a GraalVM Native Image from a modularized application, the native-image tool reads module-info.class descriptors from the module path. The ND4J module descriptors are compatible with the native-image build process. Combine the module-path setup above with the native-maven-plugin configuration described in the Maven Setup section; no additional flags are required.

Known Limitations

• OSGi: The JPMS module names follow the org.nd4j.* namespace but OSGi bundle headers (Bundle-SymbolicName, Export-Package) are separate metadata not currently shipped. OSGi integration requires a separate bundling step (e.g., bnd tool).

• Renamed internal classes: If you were using internal classes that moved as part of the split-package resolution (primarily in preset name registries), update your imports. All public APIs in org.nd4j.linalg.* and org.deeplearning4j.* are unaffected.

---

See Also

• Maven Setup — dependency management, BOM, classifier variants

• Hardware Backends — backend selection and configuration

• Snapshots — using nightly builds with Native Image