Eclipse DeepLearning4J

Eclipse Deeplearning4j is a suite of tools for running deep learning on the JVM. It's the only framework that allows you to train models from java while interoperating with the python ecosystem through a mix of python execution via our cpython bindings, model import support, and interop of other runtimes such as tensorflow-java and onnxruntime.

Consider going to our Quickstart for an overview of where to get started. If you have dependency issues please use our Required Dependencies guide.

The use cases include importing and retraining models (Pytorch, Tensorflow, Keras) models and deploying in JVM Micro service environments, mobile devices, IoT, and Apache Spark. It is a great compliment to your python environment for running models built in python, deployed to or packaged for other environments.

Deeplearning4j has several submodules including:

1. Samediff: a tensorflow/pytorch like framework for execution of complex graphs. This framework is lower level, but very flexible. It's also the base api for running onnx and tensorflow graphs.

2. Nd4j: numpy ++ for java. Contains a mix of numpy operations and tensorflow/pytorch operations.

3. Libnd4j: A lightweight, standalone c++ library enable math code to run on different devices. Optimizable for running on a wide variety of devices.

How to use this website

This website follows the layout. This website has several sections of documentation following this layout. Below is an overview of the sections of the site:

1. Multi project contains all cross project documentation such as end to end training and other whole project related documentation. This should be the default entry point for those getting started.

2. Deeplearning4j contains all of the documentation related to the core deeplearning4j apis such as the multi layer network and the computation graph. Consider this the high level framework for building neural networks. If you would like something lower level like tensorflow or pytorch, consider using samediff

3. Samediff contains all the documentation related to the samediff submodule of ND4j. Samediff is a lower level api for building neural networks similar to pytorch or tensorflow with built in automatic differentiation.

Open Source

The libraries are completely open-source, Apache 2.0 under open governance at the . The Eclipse Deeplearning4j project welcomes all contributions. See our and our to get involved.

JVM/Python/C++

Deeplearning4j can either be a compliment to your existing workflows in python and c++ or a standalone library for you to build and deploy models. Use what components you find useful.

• RBM and AutoEncoder key fixes:

  • Ensured visual bias updated and applied during pretraining.

  • RBM HiddenUnit is the activation function for this layer; thus, established derivative calculations for backprop according to respective HiddenUnit.

• RNG performance issues fixed for CUDA backend

• OpenBLAS issues fixed for macOS, powerpc, linux.

• DataVec is back to Java 7 now.

• Multiple minor bugs fixed for ND4J/DL4J

• FP16 support for CUDA

• Better performance for multi-gpu

• Including optional P2P memory access support

• Normalization support for time series and images

• Normalization support for labels

• Removal of Canova and shift to DataVec: Javadoc,

• Numerous bug fixes

• Spark improvements

All [Keras regularizers] are supported by DL4J model import:

• l1

• l2

• l1_l2

Mapping of regularizers can be found in .

• Custom layer support

• Support for custom loss functions

• Support for compressed INDArrays, for memory saving on huge data

• Initial multi-GPU support viable for standalone and Spark.

• Refactored the Spark API significantly

• Added CuDNN wrapper

Configuring the Maven build tool

You can use Deeplearning4j with Maven by adding the following to your pom.xml:

The instructions below apply to all DL4J and ND4J submodules, such as deeplearning4j-api, deeplearning4j-scaleout, and ND4J backends.

The vocabulary cache, or vocab cache, is a mechanism for handling general-purpose natural-language tasks in Deeplearning4j, including normal TF-IDF, word vectors and certain information-retrieval techniques. The goal of the vocab cache is to be a one-stop shop for text vectorization, encapsulating techniques common to bag of words and word vectors, among others.

Vocab cache handles storage of tokens, word-count frequencies, inverse-document frequencies and document occurrences via an inverted index. The InMemoryLookupCache is the reference implementation.

In order to use a vocab cache as you iterate over text and index tokens, you need to figure out if the tokens should be included in the vocab. The criterion is usually if tokens occur with more than a certain pre-configured frequency in the corpus. Below that frequency, an individual token isn't a vocab word, and it remains just a token.

We track tokens as well. In order to track tokens, do the following:

When you want to add a vocab word, do the following:

Adding the word to the index sets the index. Then you declare it as a vocab word. (Declaring it as a vocab word will pull the word from the index.)

There are two components to adding a custom layer:

1. Adding the layer configuration class: extends org.deeplearning4j.nn.conf.layers.Layer

2. Adding the layer implementation class: implements org.deeplearning4j.nn.api.Layer

The configuration layer ((1) above) class handles the settings. It's the one you would use when constructing a MultiLayerNetwork or ComputationGraph. You can add custom settings here, and use them in your layer.

The implementation layer ((2) above) class has parameters, and handles network forward pass, backpropagation, etc. It is created from the org.deeplearning4j.nn.conf.layers.Layer.instantiate(...) method. In other words: the instantiate method is how we go from the configuration to the implementation; MultiLayerNetwork or ComputationGraph will call this method when initializing the

We support all , namely:

• softmax

• elu

• selu

DL4J supports all available (except for logcosh), namely:

• mean_squared_error

• mean_absolute_error

Deeplearning4J

• Fixed issue with incorrect version dependencies in 0.9.0

• Added EmnistDataSetIterator Link

• Numerical stability improvements to LossMCXENT / LossNegativeLogLikelihood with softmax (should reduce NaNs with very large activations)

ND4J

• Added runtime version checking for ND4J, DL4J, RL4J, Arbiter, DataVec

Known Issues

• Deeplearning4j: Use of Evaluation class no-arg constructor (i.e., new Evaluation()) can result in accuracy/stats being reported as 0.0. Other Evaluation class constructors, and ComputationGraph/MultiLayerNetwork.evaluate(DataSetIterator) methods work as expected.

  • This also impacts Spark (distributed) evaluation: workaround is to replace sparkNet.evaluate(testData); with sparkNet.doEvaluation(testData, 64, new Evaluation(10))[0];, where 10 is the number of classes and 64 in the evaluation minibatch size to use.

Pre requisites

Ensure that you clone the deeplearning4j project locally.

Before importing the project, a few things of note no matter what IDE you use:

1. One submodule (libnd4j) is a c++ project that uses maven to invoke a cmake build. You may wish to edit libnd4j separately in a cmake oriented IDE like VS Code, Clion, or Eclipse c/c++. In order to build a particular nd4j backend, libnd4j should already be compiled. By default, relevant nd4j backends all look for a pre compiled libnd4j in the libnd4j directory included within the same project.

2. Maven profiles for deeplearning4j matter a lot. Especially if you want to run tests. Read more on the test profiles . For most code nd4j-tests-cpu should probably be the main profile you use.

3. Deeplearning4j uses lombok for its dependencies. Ensure you install lombok for your favorite IDE in order to use the project. Please follow the for setting this up in your IDE.

Intellij

Once cloned locally, open intellij. Please follow the guide to import from external maven sources.

Once imported, please give the project time to download associated dependencies. You can verify the status of the project in the bottom right corner.

In order to enable the project to work, the following modifications need to be made.

Shaded modules

Eclipse Deeplearning4j has a set of shaded modules. Shaded modules are artifacts that re namespace a dependency to a different location in order to use it as a set of private dependencies that do not clash with other libraries that may also share the dependency.

Intellij does not handle this very well. In order to work around this, you need to exclude all projects under the nd4j/nd4j-shade folder individually. Right click on each folder. Go to Maven -> Ignore Projects.

Assuming you follow the other steps above (lombok,libdn4j,..) then you should be able to run any module you want.

Eclipse

Note: for now the latest version of eclipse appears to fail upon first import. Any suggestions maybe reported on the .

Once cloned locally, open eclipse. Please follow the guide to import from external maven sources. Importing your project in to eclipse may take a while. Of note is due to the profile sensitive nature of the deeplearning4j suite, there maybe issues when opening and building the project.

When first finishing import of the project, a number of maven connector errors should be highlighted. Afterwards, just click resolve all later and finish. Let eclipse finish downloading sources and javadoc.

As of the latest version of eclipse, build errors may occur.

Parameters for testing

1. test.heap.size: The heap size used for maven surefire plugin sub processes

2. test.offheap.size: The off heap size used for maven surefire sub processes. This is very important for

   configuration (especially on gpu systems)

Test resources

In order to run the deeplearning4j tests, many pretrained models and other resources are required. Ensure as a dependency on your classpath. It is a big repository that needs to be mvn clean installed in order to run the tests properly. You can do this by adding -Ptestresources to your test execution when running the tests from maven.

Test profiles for enabling nd4j backends

When running deeplearning4j's tests, there are 2 main profiles to be aware of: nd4j-tests-cpu and nd4j-tests-cuda. These each enable running cpu or gpu tests respectively across the whole code base. Please ensure one of these is selected when running tests.

testresources: Used to add the test resources used for nd4j.

Test categories

Deeplearning4j uses' junit 5's tags to categorize tests in to different types. All of the tag names used throughout the code base can be found Nd4j-common-tests is included as a dependency for all tests and has a few reusable utilities used throughout the code base for tests. This makes it a great location to put common utilities we want to use throughout the code base. The tag names are mainly there to categorize tests that can take longer or use more resources so we can avoid running those dynamically depending on the size of the machine we are running tests on.

GPUs and multi threaded boxes

Note when running gpu tests on a box with more than 1 gpu, it can/will run out of memory if test.heap.size is at not at least 4g. Also of note, is when running tests

A sentence iterator is used in both Word2vec and Bag of Words.

It feeds bits of text into a neural network in the form of vectors, and also covers the concept of documents in text processing.

In natural-language processing, a document or sentence is typically used to encapsulate a context which an algorithm should learn.

A few examples include analyzing Tweets and full-blown news articles. The purpose of the sentence iterator is to divide text into processable bits. Note the sentence iterator is input agnostic. So bits of text (a document) can come from a file system, the Twitter API or Hadoop.

Depending on how input is processed, the output of a sentence iterator will then be passed to a tokenizer for the processing of individual tokens, which are usually words, but could also be ngrams, skipgrams or other units. The tokenizer is created on a per-sentence basis by a tokenizer factory. The tokenizer factory is what is passed into a text-processing vectorizer.

Some typical examples are below:

SentenceIterator iter = new LineSentenceIterator(new File("your file"));

This assumes that each line in a file is a sentence.

You can also do list of strings as sentence as follows:

This will assume that each string is a sentence (document). Remember this could be a list of Tweets or articles -- both are applicable.

You can iterate over files as follows:

This will parse the files line by line and return individual sentences on each one.

For anything complex, we recommend any pipeline that can implement more in depth support than space separated tokens.

Notes to write on: 1. Tokenizer factory interface 2. Tokenizer interface 2. How to write your own factory and tokenizer

Tokenization

What is Tokenization?

Tokenization is the process of breaking text down into individual words. Word windows are also composed of tokens. can output text windows that comprise training examples for input into neural nets, as seen here.

Example

Here's an example of tokenization done with DL4J tools:

The above snippet creates a tokenizer capable of stemming.

In Word2Vec, that's the recommended a way of creating a vocabulary, because it averts various vocabulary quirks, such as the singular and plural of the same noun being counted as two different words.

KerasEmbedding

[source]

Imports an Embedding layer from Keras.

KerasEmbedding

Pass through constructor for unit tests

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getEmbeddingLayer

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getOutputType

Get layer output type.

• param inputType Array of InputTypes

• return output type as InputType

• throws InvalidKerasConfigurationException Invalid Keras config

getNumParams

Returns number of trainable parameters in layer.

• return number of trainable parameters (1)

setWeights

Set weights for layer.

• param weights Embedding layer weights

DL4J supports all available Keras initializers, namely:

• Zeros

• Ones

• Constant

• RandomNormal

• RandomUniform

• TruncatedNormal

• VarianceScaling

• Orthogonal

• Identity

• lecun_uniform

• lecun_normal

• glorot_normal

• glorot_uniform

• he_normal

• he_uniform

The mapping of Keras to DL4J initializers can be found in .

• Added variational autoencoder

• Activation function refactor

  • Activation functions are now an interface

Configuring your build tool

While we encourage Deeplearning4j, ND4J and DataVec users to employ Maven, it's worthwhile documenting how to configure build files for other tools, like Ivy, Gradle and SBT -- particularly since Google prefers Gradle over Maven for Android projects.

The instructions below apply to all DL4J and ND4J submodules, such as deeplearning4j-api, deeplearning4j-scaleout, and ND4J backends.

The main purpose of Doc2Vec is associating arbitrary documents with labels, so labels are required. Doc2vec is an extension of word2vec that learns to correlate labels and words, rather than words with other words. Deeplearning4j's implentation is intended to serve the Java, Scala and Clojure communities.

The first step is coming up with a vector that represents the "meaning" of a document, which can then be used as input to a supervised machine learning algorithm to associate documents with labels.

In the ParagraphVectors builder pattern, the labels() method points to the labels to train on. In the example below, you can see labels related to sentiment analysis:

Here's a full working example of :

Getting started with importing Keras functional Models

Let's say you start with defining a simple MLP using Keras' functional API:

In Keras there are several ways to save a model. You can store the whole model (model definition, weights and training configuration) as HDF5 file, just the model configuration (as JSON or YAML file) or just the weights (as HDF5 file). Here's how you do each:

If you decide to save the full model, you will have access to the training configuration of the model, otherwise you don't. So if you want to further train your model in DL4J after import, keep that in mind and use model.save(...)

Getting started with importing Keras Sequential models

Let's say you start with defining a simple MLP using Keras:

In Keras there are several ways to save a model. You can store the whole model (model definition, weights and training configuration) as HDF5 file, just the model configuration (as JSON or YAML file) or just the weights (as HDF5 file). Here's how you do each:

If you decide to save the full model, you will have access to the training configuration of the model, otherwise you don't. So if you want to further train your model in DL4J after import, keep that in mind and use model.save(...) to persist your model.

KerasLocallyConnected1D

Imports a 1D locally connected layer from Keras.

KerasLocallyConnected1D

Pass-through constructor from KerasLayer

KerasBatchNormalization

Imports a BatchNormalization layer from Keras.

KerasBatchNormalization

Pass-through constructor from KerasLayer

All standard Keras optimizers are supported, but importing custom TensorFlow optimizers won't work:

• SGD

• RMSprop

• Adagrad

• Adadelta

• Adam

• Adamax

• Nadam

• TFOptimizer

Deeplearning4J has a wealth of examples of how to use its many parts. You can find the examples in the Examples Repository.

Prerequisites

The example repository consists of several separate Maven Java projects, each with their own pom files. Maven is a popular build automation tool for Java Projects. The contents of a "pom.xml" file dictate the configurations. Read more about how to configure Maven here.

Users can also refer to the simple sample project provided to get started with a clean project from scratch.

Build tools are considered standard software engineering best practice. Besides this the complexities posed by the projects in the DL4J ecosystem make dependencies too difficult to manage manually. All the projects in the DL4J ecosystem can be used with other build tools like Gradle, SBT etc. More information on that can be found .

Example Content

Projects are based on what functionality the included examples demonstrate to the user and not necessarily which library in the DL4J stack the functionality lives in.

Examples in a project are in general separated into "quickstart" and "advanced".

Each project README also lists all the examples it contains, with a recommended order to explore them in.

• This project contains a set of examples that demonstrate use of the high level DL4J API to build a variety of neural networks. Some of these examples are end to end, in the sense they start with raw data, process it and then build and train neural networks on it.

• This project contains a set of examples that demonstrate how to import Keras h5 models and TensorFlow frozen pb models into the DL4J ecosystem. Once imported into DL4J these models can be treated like any other DL4J model - meaning you can continue to run training on them or modify them with the transfer learning API or simply run inference on them.

• This project contains a set of examples that demonstrate how to do distributed training, inference and evaluation in DL4J on Apache Spark. DL4J distributed training employs a "hybrid" asynchronous SGD approach - further details can be found in the distributed deep learning documentation

Feedback & Contributions

While these set of examples don't cover all the features available in DL4J the intent is to cover functionality required for most users - beginners and advanced. File an issue if you have feedback or feature requests that are not covered here. We are also available via our for questions. We welcome contributions from the community. More information can be found We love hearing from you. Cheers!

Many more advanced models will contain custom layers, i.e. layers that aren't included in Keras.

You can import those models too, but you will have to provide an implementation of that layer yourself, as the exported model file only provides us with a name for it.

Usually, you will have found out about needing to implement a custom layer, when you saw an exception like the following:

or

Implementing a custom layer for Keras import

There are two ways of implementing a custom layer for Keras import. Which one is the right approach for you, depends on the type of layer you need to implement.

1. SameDiffLambdaLayer Use this approach if your layer doesn't have any weights and defines just a computation. It is most useful when you have to define a custom layer because you are using a lambda in your model definition. This is the approach you should be using when you've gotten the exception about no lambda layer being found.

2. KerasLayer Use this approach if your layer needs its own weights. It is most useful when you have to define some complex layer that is more than just a simple computation. This is the approach you should be using when you've gotten the exception about an unsupported layer type.

SameDiffLambdaLayer

Using a SameDiffLambdaLayer is pretty easy. You create a new class that extends it, and override the defineLayer and getOutputType methods.

This simple lambda layer just multiplies its input by 3.

After defining your layer, you have to register it to make it available on import.

The correct name for your lambda layer will depend on the model you are importing. As you, most likely, were made aware of needing to implement the lambda layer by an exception, this exception should have given you the proper name already.

KerasLayer

Implementing a full layer with weights is more complex than defining a lambda layer. You will have to create a new class that extends KerasLayer and that reads the configuration of that layer and defines it appropriately.

For examples on how this was done, take a look at and which are custom layers that were needed to be able to import GoogLeNet.

After you've defined your layer, you will have to register it to make it available on import:

Again, the appropriate name will we apparent from the exception that has notified you about needing to implement the custom layer in the first place.

KerasPReLU

[source]

Imports PReLU layer from Keras

KerasPReLU

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Invalid Keras config

getOutputType

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• param enforceTrainingConfig whether to enforce training-related configuration options

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Invalid Keras config

getPReLULayer

Get DL4J ActivationLayer.

• return ActivationLayer

setWeights

Set weights for layer.

• param weights Dense layer weights

KerasThresholdedReLU

Imports ThresholdedReLU layer from Keras

KerasThresholdedReLU

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Invalid Keras config

getOutputType

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• param enforceTrainingConfig whether to enforce training-related configuration options

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Invalid Keras config

getActivationLayer

Get DL4J ActivationLayer.

• return ActivationLayer

KerasLeakyReLU

Imports LeakyReLU layer from Keras

KerasLeakyReLU

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Invalid Keras config

getOutputType

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• param enforceTrainingConfig whether to enforce training-related configuration options

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Invalid Keras config

getActivationLayer

Get DL4J ActivationLayer.

• return ActivationLayer

KerasGaussianNoise

[source]

Keras wrapper for DL4J dropout layer with GaussianNoise.

KerasGaussianNoise

Pass-through constructor from KerasLayer

• param kerasVersion major keras version

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getOutputType

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration.

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getGaussianNoiseLayer

Get DL4J DropoutLayer with Gaussian dropout.

• return DropoutLayer

KerasAlphaDropout

Keras wrapper for DL4J dropout layer with AlphaDropout.

KerasAlphaDropout

Pass-through constructor from KerasLayer

• param kerasVersion major keras version

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getOutputType

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration.

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getAlphaDropoutLayer

Get DL4J DropoutLayer with Alpha dropout.

• return DropoutLayer

KerasGaussianDropout

Keras wrapper for DL4J dropout layer with GaussianDropout.

KerasGaussianDropout

Pass-through constructor from KerasLayer

• param kerasVersion major keras version

• throws UnsupportedKerasConfigurationException Invalid Keras config

getOutputType

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration.

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getGaussianDropoutLayer

Get DL4J DropoutLayer with Gaussian dropout.

• return DropoutLayer

KerasBidirectional

[source]

Builds a DL4J Bidirectional layer from a Keras Bidirectional layer wrapper

KerasBidirectional

Pass-through constructor from KerasLayer

• param kerasVersion major keras version

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getUnderlyingRecurrentLayer

Constructor from parsed Keras layer configuration dictionary.

• param layerConfig dictionary containing Keras layer configuration

• throws InvalidKerasConfigurationException Invalid Keras config

• throws UnsupportedKerasConfigurationException Unsupported Keras config

getBidirectionalLayer

Get DL4J Bidirectional layer.

• return Bidirectional Layer

getOutputType

Get layer output type.

• param inputType Array of InputTypes

• return output type as InputType

• throws InvalidKerasConfigurationException Invalid Keras config

getNumParams

Returns number of trainable parameters in layer.

• return number of trainable parameters

getInputPreprocessor

Gets appropriate DL4J InputPreProcessor for given InputTypes.

• param inputType Array of InputTypes

• return DL4J InputPreProcessor

• throws InvalidKerasConfigurationException Invalid Keras configuration exception

• see org.deeplearning4j.nn.conf.InputPreProcessor

setWeights

Set weights for Bidirectional layer.

• param weights Map of weights

• UI overhaul: new training UI has considerably more information, supports persistence (saving info and loading later), Japanese/Korean/Russian support. Replaced Dropwizard with Play framework.

• Import of models configured and trained using

  • Imports both Keras model

ND4J works atop so-called backends, or linear-algebra libraries, such as Native nd4j-native and nd4j-cuda-10.2 (GPUs), which you can select by pasting the right dependency into your project’s POM.xml file.

ND4J backends for GPUs and CPUs

You can choose GPUs or native CPUs for your backend linear algebra operations by changing the dependencies in ND4J's POM.xml file.

For CUDA we usually support the 2 most recent cuda versions for a given release.

For M2.1, we support cuda 11.4 and 11.6.

What is AVX, and why does it matter?

AVX (Advanced Vector Extensions) is a set of CPU instructions for accelerating numerical computations. See for more details.

Note that AVX only applies to nd4j-native (CPU) backend for x86 devices, not GPUs and not ARM/PPC devices.

Why AVX matters: performance. You want to use the version of ND4J compiled with the highest level of AVX supported by your system.

AVX support for different CPUs - summary:

Highlights

A number of bug fixes following the M1 release, thanks to the feedback from the community, allowed us to quickly sort out a few issues. This is a minor bug fix release to address short comings found with M1. Most fixes were related to keras import, the cnn/rnn helpers, and python4j.

Snapshots will also be published every 2 days automatically now https://github.com/eclipse/deeplearning4j/pull/9355 to get around sonatype ossrh deletion of snapshots every 3 days. This should increase robustness of the snapshots.

Worked around an issue with github actions pre emptively upgrading visual studio breaking the cuda builds: https://github.com/eclipse/deeplearning4j/pull/9364

Added backwards compatibility for centos 6 via a new linux-x86_64-compat classifier enabling use of older glibcs on centos 7:

A number of bugs were fixed with LSTM and CUDNN:

Known issues

- avoid shuffle operations on gpu. Pre save data on cpu in mini batches. For more help, please post on the forums at https://community.konduit.ai/

Deeplearning4j

Features and Enhancements

• Add batch normalization support for RNNs:

• Disable old helpers by default

• Minor unit test fixes:

• Add keras support for cnn 1d NWHC:

Bug fixes

• Fixed an issue with helper reflection ensuring the classes would be loaded properly

• Fix minor workspace activation bug:

• Fixed compilation error when running anything more than jdk 8 and NIO buffers:

Nd4j

Features and Enhancements

• Add Eigen op as public ensuring easier use when running eigenvalue decomposition

Bug fixes

• Fixes minor issue with choice(..) op thanks to

• Minor applyScalar typo fix:

Datavec

Features and Enhancements

Bug fixes

Fixed serialization bug with StringToTimeTransform: thanks to community member

Python4j

Features and Enhancements

• Made python4j's python path setting more robust by migrating from set path calls to add path calls:

Bug fixes

• Fixes bug with numpy import array jvm crashes:

Samediff

Features and Enhancements

Bug fixes

Fixed inconsistent conventions between SameDiffVariable getArr and getArrForName()..

Highlights

In light of the coming 1.0, the project has decided to cut a number of modules before the final release. These modules have not had many users in the past and have created confusion for many users just trying to use a few simple apis. Many of these modules have not been maintained.

There will likely be 1 or 2 more milestone releases before the final 1.0. These should be considered checkpoints.

These modules include:

1. Arbiter

2. Jumpy

3. Datavec modules for video, audio, audio, sound. The computer vision datavec module

   will continue to be available.

4. Tokenizers: The tokenizers for chinese, japanese, korean were imported from other frameworks

   and not really updated.

5. Scalnet, Nd4s: We removed the scala modules due to the small user base. We welcome 3rd party enhancements

   to the framework for syntatic sugar such as kotlin and scala. The framework's focus will be on providing

   the underlying technology rather than the defacto interfaces. If there is interest in something higher level, please discuss it on

ARM support: We have included armcompute modules for core convolution routines. These routines can be found

TVM: We now support running TVM modules. Docs coming soon.

We've updated our shaded modules to newer versions to mitigate security risks. These modules include: 1. jackson 2. guava

Cuda 11: We've upgraded dl4j and associated modules to support cuda 11 and 11.2.

A more modular model import framework supporting tensorflow and onnx: 1. Model mapping procedures loadable as protobuf 2. Defining custom rules for import to work around unsupported or custom layers/operations 3. Op descriptor for all operations in nd4j

This will enable users to override model import behavior to run their own custom models. This means, in most circumstances, there will be no need to modify model import core code anymore. Instead, users will be able to provide definitions and custom rules for their graphs.

Users will be expected to convert their models in an external process. This means running standalone conversions for their models. This extends to keras import as well. Sometimes users convert their models in production directly from keras.

The workflow going forward is to ensure that your model is converted ahead of time to avoid performance issues with converting large models.

Removed ppc from nd4j-native-platform and nd4j-cuda-platform. If you need this architecture, please contact us or build from source.

Added more support for avx/mkldnn/cudnn linked acceleration in our c++ library. We now have the ability to distribute more combinations of pre compiled math kernels via different combinations of classifiers. See the for more details.

. This is useful for OSGI and application server environments.

We've upgraded arrow to 4.0.0 enabling the associated nd4j-arrow and datavec-arrow modules to be used without netty clashes.

Deeplearning4j

Bug fixes

• Improved keras model import support for NWHC as well as NCHW input formats for both rnn and cnn

Nd4j

Features and Enhancements

• : We now have basic support for CTC loss in nd4j. This will enable the import of CTC loss based models for speech recognition as well as OCR.

Bug fixes

Python4j

Features and Enhancements

Rewritten and more stable python execution. This allows better support for multi threaded environments.

Bug fixes

Contributors:

Prerequisites

Before contributing, make sure you know the structure of all of the Eclipse Deeplearning4j libraries. As of early 2018, all libraries now live in the Deeplearning4j monorepo. These include:

• DeepLearning4J: Contains all of the code for learning neural networks, both on a single machine and distributed.

• ND4J: “N-Dimensional Arrays for Java”. ND4J is the mathematical backend upon which DL4J is built. All of DL4J’s neural networks are built using the operations (matrix multiplications, vector operations, etc) in ND4J. ND4J is how DL4J supports both CPU and GPU training of networks, without any changes to the networks themselves. Without ND4J, there would be no DL4J.

• DataVec: DataVec handles the data import and conversion side of the pipeline. If you want to import images, video, audio or simply CSV data into DL4J: you probably want to use DataVec to do this.

• RL4J: Reinforcement Learning for Java. This set of libraries contains the ability to do reinforcement learning built on the deeplearning4j library.

• Samediff: Built within the nd4j library, this library contains a tensorflow/pytorch like library for building data flow graphs.

We also have an extensive examples repository at .

Ways to contribute

There are numerous ways to contribute to DeepLearning4J (and related projects), depending on your interests and experince. Here’s some ideas:

• Add new types of neural network layers (for example: different types of RNNs, locally connected networks, etc)

• Add a new training feature

• Bug fixes

• DL4J examples: Is there an application or network architecture that we don’t have examples for?

There are a number of different ways to find things to work on. These include:

• Looking at the issue trackers:

• Reviewing our Roadmap

• Talking to the developers on the

General guidelines

Before you dive in, there’s a few things you need to know. In particular, the tools we use:

• Maven: a dependency management and build tool, used for all of our projects. See this for details on Maven.

• Git: the version control system we use

• Project Lombok: Project Lombok is a code generation/annotation tool that is aimed to reduce the amount of ‘boilerplate’ code (i.e., standard repeated code) needed in Java. To work with source, you’ll need to install the Project Lombok plugin for your IDE

Things to keep in mind:

• Code should be Java 7 compliant

• If you are adding a new method or class: add JavaDocs

• You are welcome to add an author tag for significant additions of functionality. This can also help future contributors, in case they need to ask questions of the original author. If multiple authors are present for a class: provide details on who did what (“original implementation”, “added feature x” etc)

This page explains steps required to contribute code to the projects in the eclipse/deeplearning4j GitHub repository: https://github.com/eclipse/deeplearning4j

Contributors (anyone who wants to commit code to the repository) need to do two things, before their code can be merged:

1. Sign the Eclipse Contributor Agreement (once)

2. Sign commits (each time)

Why Is This Required?

These two requirements must be satisfied for all Eclipse Foundation projects, not just DL4J and ND4J. A full list of Eclipse Foundation Projects can be found here:

By signing the ECA, you are essentially asserting that the code you are submitting is something that either you wrote, or that you have the right to contribute to the project. This is a necessary legal protection to avoid copyright issues.

By signing your commits, you are asserting that the code in that particular commit is your own.

Signing the Eclipse Contributor Agreement

You only need to sign the Eclipse Contributor Agreement (ECA) once. Here's the process:

Step 1: Sign up for an Eclipse account

This can be done at

Note: You must register using the same email as your GitHub account (the GitHub account you want to submit pull requests from).

Step 2: Sign the ECA

Go to and follow the instructions.

Signing Your Commits

Signing a New Commit

There are a few ways to sign commits. Note that you can use any of these aoptions.

Option 1: Use -s When Committing on Command Line

Signing commits here is simple:

Note the use of -s (lower case s) - upper-case S (i.e., -S) is for GPG signing (see below).

Option 2: Set up Bash Alias (or Windows cmd Alias) for Automated Signing

For example, you could set up the following alias in Bash:

Then committing would be done with the following:

For Windows command line, similar options are available through a few mechanisms (see )

One simple way is to create a gcm.bat file with the following contents, and add it to your system path:

You can then commit using the same process as above (i.e., gcm "My Commit")

Option 3: Use GPG Signing

For details on GPG signing, see

Note that this option can be combined with aliases (above), as in alias gcm='git commit -S -m' - note the upper case -S for GPG signing.

Option 4: Commit using IntelliJ with Auto Signing

IntelliJ can be used to perform git commits, including through signed commits. See for details.

Checking If A Commit Is Signed

After performing a commit, you can check in a few different ways. One way is to use git log --show-signature -1 to show the signature for the last commit (use -5 to show the last 5 commits, for example)

The output will look like:

The top commit is unsigned, and the bottom commit is signed (note the presence of the Signed-off-by).

If You Forget to Sign a Commit - Amending the Last Commit

If you forgot to sign the last commit, you can use the following command:

If You Forget to Sign Multiple Commits

Suppose your branch has 3 new commits, all of which are unsigned:

One simple way is to squash and sign these commits. To do this for the last 3 commits, use the following: (note you might want to make a backup first)

The result:

You can confirm that the commit is signed using git log -1 --show-signature as shown earlier.

Note that your commits will be squashed once they are merged to master anyway, so the loss of the commit history does not matter.

If you are updating an existing PR, you may need to force push using -f (as in git push X -f).

Deeplearning4j has several steps to a release. Below is a brief outline with follow on descriptions.

1. Compile libnd4j for different cpu architectures

2. Ensure the current javacpp dependencies such as python, mkldnn, cuda, .. are up to date

3. Run all integration tests on core platforms (windows, mac, linux) with both cpu and gpu

4. Create a staging repository for testing using github actions running manually on each platform

5. Update the examples to be compatible with the latest release

6. Run the deeplearning4j-examples as a litmus tests on all platforms (including embedded)

   to sanity check platform specific numerical bugs using the staging repository

7. Double check any user related bugs to see if they should block a release

8. Hit release button

9. Perform follow up release of -platform projects under same version

10. Tag release

Compile libnd4j on different cpu architectures

Compiling libnd4j on different cpu architectures ensures there is platform optimized math in c++ for each platform. The is a self contained cmake project that can be run on different platforms. In each there are steps for deploying for each platform.

At the core of compiling from source for libnd4j is a maven pom.xml that is run as part of the overall build process that invokes our with various parameters that then get passed to our overall cmake structure for compilation. This script exists to formalize some of the required parameters for invokving cmake. Any developer is welcome to invoke cmake directly.

• Platform compatibility

  We currently compile libnd4j on ubuntu 16.04. This means glibc 2.23.

  For our cuda builds, we use gcc7.

  Users of older glibc versions may need to compile from source. For our standard release, we try to keep it reasonably old, but do not support end of lifed

  end of linux distributions for public builds.

• Platform specific helpers

Each build of libnd4j links against an accelerated backend for and convolution operations such as , , or The implementations for each platform can be found

Ensure the current javacpp dependencies such as python, mkldnn, cuda, .. are up to date

This is a step that just ensures that the dl4j release matches the current state of the dependencies provided by javacpp on maven central. This affects every module including python4j, nd4j-native/cuda, datavec-image, among others. The versions of everything can be found in the top level The general convention is library version followed by a - and the version of javacpp that that version uses.

Of note here is that certain older versions of libraries can use older javacpp versions. It is recommended that that the desired version be up to date if possible. Otherwise, if an older version of javacpp is the only version available, this is generally ok.

Run all integration tests on core platforms (windows, mac, linux) with both cpu and gpu

We run all of the major integration tests on the core major platforms where higher end compute is accessible. This is generally a bigger machine. It is expected that some builds can take up to 2 hours depending on the specs of the desired machine.

This step may also involve invoking tests with specific tags if only running a subset of tests is desired. This can be achived using the -Dgroups flag.

Update the examples to be compatible with the latest release

To ensure the examples stay compatible with the current release, we also tag the release version to be the latest version found on maven central. This step may also involve adding or removing examples for new or deprecated features respectivley.

Ensure different classifiers work

1. Different supported cuda versions with and without cudnn

2. Onednn and associated classifiers per platform

Android

Ensure testing happens on the android emulator.

Run the deeplearning4j-examples as a litmus tests on all platforms (including embedded)

The examples contain a set of tests which just allow us to run maven clean test on a small number of examples. Instead of us picking examples manually, we can just run mvn clean test on any platform we need by just specifying a version of dl4j to depend on and usually a

Double check any user related bugs to see if they should block a release

Generally, sometimes users will raise issues right before a release that can be critical. It is the sole discretion of the maintainers to ask the user to use snapshots or to wait for a follow on version. For certain fixes, we will publish quick bugfix releases. If your team has specific requirements on a release, please contact us on the

Hit release button

This means after , hitting the release button initiating a sync of the staging repository with the desired version to maven central. Sync usually takes 2 hours or less.

Ensure a tag exists

After a release happens, a version update to the stable version + a github tag needs to happen. This is achived in the desktop app by going to: 1. History 2. Right click on target commit you want to tag 3. Click tag 4. Push the revision 5. Update the version back to snapshot after tag.

The DL4J suite has different configuration requirements for your dependencies depending on your use case. This page gives an overview of common cases to ensure people have what they need to get started.

Quick Overview

The deeplearning4j suite has a few common components to consider. Most users just need the following dependencies:

1. Deeplearning4j NN: https://search.maven.org/artifact/org.deeplearning4j/deeplearning4j-nn/1.0.0-M2.1/jar This contains the DSL for running the simpler neural networks.

2. A is also required. This is for running different code on cpus/gpus.

3. The deeplearning4j suite uses for running platform specific native code. These dependencies will have classifiers for different platforms. Read up on classifiers at baeldung: See a list here:

   1. Each jar has a classifier for different platforms. These platform specific jars only contain native code for different platforms. The classifiers come in the form of OS-architecture-platform-helper-optimization

      1. OS: This is linux,windows, mac, android
4. All versions must be the same dependencies. We do not support mixing versions. There is no reason to. Each version signifies what is compatible API wise.

Typical Problems and how to fix

1. Very large jar files: Your jar will be over sized and contain a bunch of dependencies. These are cross platform dependencies. Each platform dependency contains native c++ binaries for a specific platform. This keeps binaries smaller for different platforms but adds complexity for deployment. Simplify this by either using -platform dependencies with your specific platform in mind or specify the dependencies manually. This means including a dependency without the platform (this contains the actual java code for the dependency) and the dependency with classifier.

2. GLIBC issues: Depending on the platform, some users may need to run older glibc versions. If you get GLIBC issues, please use the linux-x86_64-compat dependency. If you need a custom build, please contact us for support. https://konduit.ai/

3. Cuda version clashes: Users may run in to clashing dependencies with cuda. Ensure you only have 1 cuda install. On linux you can also use javacpp's cuda redist artifacts for different versions of cuda. Note these dependencies are very large. See an overview here for the various redist artifacts:

If you are looking for more advanced neural networks, we recommend this just needs the nd4j-api and

Computer Vision

Computer vision workloads typically need deeplearning4j-nn and

NLP

If you are looking to run NLP workloads, you just need deeplearning4j-nlp and a

Spark

Use dl4j-spark_2.12 and a - note also depending on your spark job you may run in to jar file size limits. Ensure you minimize your dependencies as much as possible. In this case the dependencies should be restricted to the specific platform the jar will be running on.

Android

Users running android should be heavily aware of different ABIs as mentioned above. We recommend using the nd4j-minimizer backend to avoid dependencies on openblas if they are not needed. Note only cpu based android calculations are supported.

Using Deeplearning4j with cuDNN

There are 2 ways of using cudnn with deeplearning4j. One is an older way described below that is built in to the various deeplearning4j layers at the java level.

The other is to use the new nd4j cuda bindings that link to cudnn at the c++ level. Both will be described below. The newer way first, followed by the old way.

Cudnn setup

The actual library for cuDNN is not bundled, so be sure to download and install the appropriate package for your platform from NVIDIA:

Note there are multiple combinations of cuDNN and CUDA supported. Deeplearning4j's cuda support is based on . The way to read the versioning is: cuda version - cudnn version - javacpp version. For example, if the cuda version is set to 11.2, you can expect us to support cudnn 8.1.

To install, simply extract the library to a directory found in the system path used by native libraries. The easiest way is to place it alongside other libraries from CUDA in the default directory (/usr/local/cuda/lib64/ on Linux, /usr/local/cuda/lib/ on Mac OS X, and C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin\, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\, or C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin\ on Windows).

Alternatively, in the case of the most recent supported cuda version, cuDNN comes bundled with the "redist" package of the . , we can add the following dependencies instead of installing CUDA and cuDNN:

The same versioning scheme for redist applies to the cuda bindings that leverage an installed cuda.

Using cuDNN via nd4j

Similar to our avx bindings, nd4j leverages our c++ library libnd4j for running mathematical operations. In order to use cudnn, all you need to do is change the cuda backend dependency from:

or for cuda 11.4:

to

or for cuda 11.0:

For jetson nano cuda 10.2:

For windows (note: all we did was change linux to windows, the same cuda versions are applicable here for linux as well as windows):

Note that we are only adding an additional dependency. The reason we use an additional classifier is to pull in an optional dependency on cudnn based routines. The default does not use cudnn, but instead built in standalone routines for various operations implemented in cudnn such as conv2d and lstm.

For users of the -platform dependencies such as nd4j-cuda-11.2-platform, this classifier is still required. The -platform dependencies try to set sane defaults for each platform, but give users the option to include whatever they want. If you need optimizations, please become familiar with this.

Memory Management for ND4J/DL4J: How does it work?

ND4J uses off-heap memory to store NDArrays, to provide better performance while working with NDArrays from native code such as BLAS and CUDA libraries.

"Off-heap" means that the memory is allocated outside of the JVM (Java Virtual Machine) and hence isn't managed by the JVM's garbage collection (GC). On the Java/JVM side, we only hold pointers to the off-heap memory, which can be passed to the underlying C++ code via JNI for use in ND4J operations.

To manage memory allocations, we use two approaches:

• JVM Garbage Collector (GC) and WeakReference tracking

• MemoryWorkspaces - see for details

Despite the differences between these two approaches, the idea is the same: once an NDArray is no longer required on the Java side, the off-heap associated with it should be released so that it can be reused later. The difference between the GC and MemoryWorkspaces approaches is in when and how the memory is released.

• For JVM/GC memory: whenever an INDArray is collected by the garbage collector, its off-heap memory will be deallocated, assuming it is not used elsewhere.

• For MemoryWorkspaces: whenever an INDArray leaves the workspace scope - for example, when a layer finished forward pass/predictions - its memory may be reused without deallocation and reallocation. This results in better performance for cyclical workloads like neural network training and inference.

Configuring Memory Limits

With DL4J/ND4J, there are two types of memory limits to be aware of and configure: The on-heap JVM memory limit, and the off-heap memory limit, where NDArrays live. Both limits are controlled via Java command-line arguments:

• -Xms - this defines how much memory JVM heap will use at application start.

• -Xmx - this allows you to specify JVM heap memory limit (maximum, at any point). Only allocated up to this amount (at the discretion of the JVM) if required.

• -Dorg.bytedeco.javacpp.maxbytes - this allows you to specify the off-heap memory limit. This can also be a percentage, in which case it would apply to maxMemory.

Example: Configuring 1GB initial on-heap, 2GB max on-heap, 8GB off-heap, 10GB maximum for process:

Gotchas: A few things to watch out for

• With GPU systems, the maxbytes and maxphysicalbytes settings currently also effectively defines the memory limit for the GPU, since the off-heap memory is mapped (via NDArrays) to the GPU - read more about this in the GPU-section below.

• For many applications, you want less RAM to be used in JVM heap, and more RAM to be used in off-heap, since all NDArrays are stored there. If you allocate too much to the JVM heap, there will not be enough memory left for the off-heap memory.

• If you get a "RuntimeException: Can't allocate [HOST] memory: xxx; threadId: yyy", you have run out of off-heap memory. You should most often use a WorkspaceConfiguration to handle your NDArrays allocation, in particular in e.g. training or evaluation/inference loops - if you do not, the NDArrays and their off-heap (and GPU) resources are reclaimed using the JVM GC, which might introduce severe latency and possible out of memory situations.

Memory-mapped files

ND4J supports the use of a memory-mapped file instead of RAM when using the nd4j-native backend. On one hand, it's slower then RAM, but on other hand, it allows you to allocate memory chunks in a manner impossible otherwise.

Here's sample code:

In this case, a 1GB temporary file will be created and mmap'ed, and NDArray x will be created in that space. Obviously, this option is mostly viable for cases when you need NDArrays that can't fit into your RAM.

GPUs

When using GPUs, oftentimes your CPU RAM will be greater than GPU RAM. When GPU RAM is less than CPU RAM, you need to monitor how much RAM is being used off-heap. You can check this based on the JavaCPP options specified above.

We allocate memory on the GPU equivalent to the amount of off-heap memory you specify. We don't use any more of your GPU than that. You are also allowed to specify heap space greater than your GPU (that's not encouraged, but it's possible). If you do so, your GPU will run out of RAM when trying to run jobs.

We also allocate off-heap memory on the CPU RAM as well. This is for efficient communicaton of CPU to GPU, and CPU accessing data from an NDArray without having to fetch data from the GPU each time you call for it.

If JavaCPP or your GPU throw an out-of-memory error (OOM), or even if your compute slows down due to GPU memory being limited, then you may want to either decrease batch size or increase the amount of off-heap memory that JavaCPP is allowed to allocate, if that's possible.

Try to run with an off-heap memory equal to your GPU's RAM. Also, always remember to set up a small JVM heap space using the Xmx option.

Note that if your GPU has < 2g of RAM, it's probably not usable for deep learning. You should consider using your CPU if this is the case. Typical deep-learning workloads should have 4GB of RAM at minimum. Even that is small. 8GB of RAM on a GPU is recommended for deep learning workloads.

It is possible to use HOST-only memory with a CUDA backend. That can be done using workspaces.

Example:

It's not recommended to use HOST-only arrays directly, since they will dramatically reduce performance. But they might be useful as in-memory cache pairs with the INDArray.unsafeDuplication() method.

All Keras constraints are supported:

• max_norm

• non_neg

• unit_norm

• min_max_norm

Mapping Keras to DL4J constraints happens in .

Highlights

Significant changes for views being used within nd4j and samediff via the create_view op allow for users to directly leverage views in combination with graph operations reducing allocations and allowing for increased performance.

Cuda 11.4 and 11.6 added. Add nd4j-cuda-11.4 and nd4j-cuda-11.6 to your dependencies.

New Onnx ops support added ():

Deeplearning4J

• Workspaces feature added (faster training performance + less memory)

• SharedTrainingMaster added for Spark network training (improved performance) ,

• ParallelInference added - wrapper that server inference requests using internal batching and queues

0.8.0 -> 0.9.0 Transition Notes

Deeplearning4j

• Updater configuration methods such as .momentum(double) and .epsilon(double) have been deprecated. Instead: use .updater(new Nesterovs(0.9)) and .updater(Adam.builder().beta1(0.9).beta2(0.999).build())

Overview

Every machine-learning workflow consists of at least two parts. The first is loading your data and preparing it to be used for learning. We refer to this part as the ETL (extract, transform, load) process. DataVec is the library we built to make building data pipelines easier. The second part is the actual learning system itself. That is the algorithmic core of DL4J.

All deep learning is based on vectors and tensors, and DL4J relies on a tensor library called ND4J. It provides us with the ability to work with n-dimensional arrays (also called tensors). Thanks to its different backends, it even enables us to use both CPUs and GPUs.

What are workspaces?

ND4J offers an additional memory-management model: workspaces. That allows you to reuse memory for cyclic workloads without the JVM Garbage Collector for off-heap memory tracking. In other words, at the end of the workspace loop, all INDArrays' memory content is invalidated. Workspaces are integrated into DL4J for training and inference.

The basic idea is simple: You can do what you need within a workspace (or spaces), and if you want to get an INDArray out of it (i.e. to move result out of the workspace), you just call INDArray.detach() and you'll get an independent INDArray

Deeplearning4j: Keras model import

provides routines for importing neural network models originally configured and trained using , a popular Python deep learning library.

Once you have imported your model into DL4J, our full production stack is at your disposal. We support import of all Keras model types, most layers and practically all utility functionality. Please check for a complete list of supported Keras features.

Note to users: tf.keras models are also supported. Please check for an overview of what to expect for tf.keras as well as other features. Our documentation needs to be updated to reflect the changes between keras and tf.keras. For now, users should aware of this as you read the below docs. Migrating from keras to tf.keras mainly involves changing the imports in your python script. The equivalent kind of changes needed to happen for the model import in deeplearning4j. Those changes happened in beta7.

DL4J and Javacpp overview

DL4J heavily depends on for its interop between java and platform optimized c++ libraries. However, due to our usage of JNI this comes with certain complexities in the build anyone should be aware of.

The following modules rely on javacpp as part of their build process: 1. nd4j-native 2. nd4j-native-presets 3. nd4j-cuda 4. nd4j-cuda-presets

Each of these libraries are what comprise our nd4j backends. Leveraging [libnd4j], javacpp handles linking each nd4j-backend against the libnd4j c++ codebase. This linking is done using a libnd4j home. This will contain all of the include files and necessary binary files for specific platforms. By default, nd4j backends and the libnd4j code base are compiled within the same build step. This is the recommended default, but for specific circumstances. A libnd4j release is also uploaded to maven central as a zip file and can be used in place of libnd4j compilation. See our

What is early stopping?

When training neural networks, numerous decisions need to be made regarding the settings (hyperparameters) used, in order to obtain good performance. Once such hyperparameter is the number of training epochs: that is, how many full passes of the data set (epochs) should be used? If we use too few epochs, we might underfit (i.e., not learn everything we can from the training data); if we use too many epochs, we might overfit (i.e., fit the 'noise' in the training data, and not the signal).

Early stopping attempts to remove the need to manually set this value. It can also be considered a type of regularization method (like L1/L2 weight decay and dropout) in that it can stop the network from overfitting.

The idea behind early stopping is relatively simple: