Nd4j allows execution of models via onnx runtime using nd4j's INDArray as a data structure. Leveraging the nd4j-onnxruntime interop is fairly simple. An onnx model can be loaded and executed as follows:

        File f = new File ("path/to/your/model.onnx");
        INDArray x = Nd4j.scalar(1.0f).reshape(1,1);
        INDArray y = Nd4j.scalar(1.0f).reshape(1,1);
        OnnxRuntimeRunner onnxRuntimeRunner = OnnxRuntimeRunner.builder()
                .modelUri(f.getAbsolutePath())
                .build();
        Map<String,INDArray> inputs = new LinkedHashMap<>();
        inputs.put("x",x);
        inputs.put("y",y);
        Map<String, INDArray> exec = onnxRuntimeRunner.exec(inputs);
        INDArray z = exec.get("z");

This outputs a result with a map of output names to the ndarray result from onnx.

In maven, add the following dependency:

<dependency>
   <groupId>org.nd4j</groupId>
   <artifactId>nd4j-onnxruntime</groupId>
   <version>${nd4j.version}</version>
</dependency>

Note, this depends on javacpp's onnxruntime bindings. This means onnx runtime's native binaries are managed by javacpp. Javacpp will bundle all native binaries for all platforms by default unless you specify a platform. You can do this by specifying a -Dplatform=your-platform-of-choice. You may find more here in the javacpp docs.

Nd4j allows execution of models via javacpp's tvm bindings using nd4j's INDArray as a data structure. Leveraging the nd4j-tvm interop is fairly simple.

A TVM model can be loaded and executed as follows:

This outputs a result with a map of output names to the ndarray result from tvm.

In maven, add the following dependency:

Note, this depends on javacpp's tvm bindings. This means tvm's native binaries are managed by javacpp. Javacpp will bundle all native binaries for all platforms by default unless you specify a platform. You can do this by specifying a -Dplatform=your-platform-of-choice. You may find more in the javacpp docs.

Nd4j allows execution of models via javacpp's tensorflow bindings using nd4j's INDArray as a data structure. Leveraging the nd4j-tensorflow interop is fairly simple.

Note, this is based on tensorflow 1.x. TF java 2 will be coming at a later time.

A tensorflow model can be loaded and executed as follows:

        File f = new File ("path/to/your/model.pb");
        INDArray x = Nd4j.scalar(1.0f).reshape(1,1);
        INDArray y = Nd4j.scalar(1.0f).reshape(1,1);
        List<String> inputNames = Arrays.asList("x","y");
        List<String> outputNames = Arrays.asList("z");
        GraphRunner tensorflowRunner = GraphRunner.builder()
                .inputNames(inputNames)
                .graphPath(f)
                .outputNames(outputNames).
                build();
        Map<String,INDArray> inputs = new LinkedHashMap<>();
        inputs.put("x",x);
        inputs.put("y",y);
        Map<String, INDArray> exec = tensorflowRunner.exec(inputs);
        INDArray z = exec.get("z");

This outputs a result with a map of output names to the ndarray result from tensorflow.

In maven, add the following dependency:

<dependency>
   <groupId>org.nd4j</groupId>
   <artifactId>nd4j-tensorflow</groupId>
   <version>${nd4j.version}</version>
</dependency>

Note, this depends on javacpp's tensorflow bindings. This means tensorflow's native binaries are managed by javacpp. Javacpp will bundle all native binaries for all platforms by default unless you specify a platform. You can do this by specifying a -Dplatform=your-platform-of-choice. You may find more here in the javacpp docs.