Highlights - 1.0.0-beta Release

    Performance and memory optimizations for DL4J


Deeplearning4J: New Features

    New or enhanced layers:
      Added Cropping1D layer Link
      Added Convolution3D, Cropping3D, UpSampling3D, ZeroPadding3D, Subsampling3D layers (all with Keras import support): Link Link
      Added EmbeddingSequenceLayer (EmbeddingLayer for time series) Link
      Added OCNNOutputLayer (one-class neural network) - implementation of this paper - Link
      Added FrozenLayerWithBackprop layer Link
      Added DepthwiseConvolution2D layer Link
    Added ComputationGraph.output(DataSetIterator) method Link
    Added MultiLayerNetwork/ComputationGraph.layerInputSize methods Link Link
    Added SparkComputationGraph.feedForwardWithKey overload with feature mask support Link
    Added MultiLayerNetwork.calculateGradients method (for easily getting parameter and input gradients, for example for some model interpretabilithy approaches) Link Link
    Added support to get input/activation types for each layer from configuration: ComputationGraphConfiguration.getLayerActivationTypes(InputType...), ComputationGraphConfiguration.GraphBuilder.getLayerActivationTypes(), NeuralNetConfiguration.ListBuilder.getLayerActivationTypes(), MultiLayerConfiguration.getLayerActivationTypes(InputType) methods Link
    Evaluation.stats() now prints confusion matrix in easier to read matrix format, rather than list format Link
    Added ModelSerializer.addObjectToFile, .getObjectFromFile and .listObjectsInFile for storing arbitrary Java objects in same file as saved network Link
    Added SpatialDropout support (with Keras import support) Link
    Added MultiLayerNetwork/ComputationGraph.fit((Multi)DataSetIterator, int numEpochs) overloads Link
    Added performance (hardware) listeners: SystemInfoPrintListener and SystemInfoFilePrintListener Link

Deeplearning4J: Bug Fixes and Optimizations

    Performance and memory optimizations via optimizations of internal use of workspaces Link
    Reflections library has entirely been removed from DL4J and is no longer required for custom layer serialization/deserialization Link, Link
      Fixes issues with custom and some Keras import layers on Android
    RecordReaderMultiDataSetIterator will no longer try to convert unused columns to numerical values Link
    Added new model zoo models:
      (to do)
    Fixes for Android compilation (removed duplicate classes, aligned versions, removed some dependencies) Link Link Link
    Fix for RecordReaderMulitDataSetIterator where output could be incorrect for some constructors Link
    Non-frozen layers before a frozen layer will no longer be skipped during backprop (useful for GANs and similar architectures) Link Link
    Fixed issue where ComputationGraph topological sort may not be consistent on all platforms; could sometimes break ComputationGraphs (with multiple valid topological orderings) trained on PC and deployed on Android Link
    Fixed issue with CuDNN batch norm using 1-decay instead of decay Link
    deeplearning4j-cuda no longer throws exceptions if present on classpath with nd4j-native backend set to higher priority Link
    Added RNG control for CifarDataSetIterator Link
    WordVectorSerializer now deletes temp files immediately once done Link

Deeplearning4J: API Changes (Transition Guide): 1.0.0-alpha to 1.0.0-beta

    WorkspaceMode.SINGLE and SEPARATE have been deprecated; use WorkspaceMode.ENABLED instead
    Internal layer API changes: custom layers will need to be updated to the new Layer API - see built-in layers or custom layer example
    Custom layers etc in pre-1.0.0-beta JSON (ModelSerializer) format need to be registered before they can be deserialized due to JSON format change. Built-in layers and models saved in 1.0.0-beta or later do not require this. Use NeuralNetConfiguration.registerLegacyCustomClassesForJSON(Class) for this purpose
    IterationListener has been deprecated in favor of TrainingListener. For existing custom listeners, switch from implements TrainingListener to extends BaseTrainingListener Link
    ExistingDataSetIterator has been deprecated; use fit(DataSetIterator, int numEpochs) method instead

Deelpearning4J: 1.0.0-beta Known Issues

    ComputationGraph TrainingListener onEpochStart and onEpochEnd methods are not being called correctly
    DL4J Zoo Model FaceNetNN4Small2 model configuration is incorrect, causing issues during forward pass
    Early stopping score calculators with values thar should be maximized (accuracy, f1 etc) are not working properly (values are minimized not maximized). Workaround: override ScoreCalculator.calculateScore(...) and return 1.0 - super.calculateScore(...).

Deeplearing4J: Keras Import

Deeplearning4J: Keras Import - API Changes (Transition Guide): 1.0.0-alpha to 1.0.0-beta


ND4J: New Features

ND4J: Known Issues

    Not all op gradients implemented for automatic differentiation
    Vast majority of new operations added in 1.0.0-beta do NOT use GPU yet.

ND4J: API Changes (Transition Guide): 1.0.0-alpha to 1.0.0-beta


DataVec: New Features

    ImageRecordReader now logs number of inferred label classes (to reduce risk of users missing a problem if something is misconfigured) Link
    Added AnalyzeSpark.getUnique overload for multiple columns Link
    Added performance/timing module Link

DataVec: Optimizations and Bug Fixes

    Reduced ImageRecordReader garbage generation via buffer reuse Link
    Fixes for Android compilation (aligned versions, removed some dependencies) Link Link
    Removed Reflections library use in DataVec Link
    Fix for TransformProcessRecordReader batch support Link
    Fix for TransformProcessRecordReader with filter operations Link
    Fixed issue with ImageRecordReader/ParentPathLabelGenerator incorrectly filtering directories containing . character(s) Link
    ShowImageTransform now initializes frame lazily to avoid blank windows Link

DataVec: API Changes (Transition Guide): 1.0.0-alpha to 1.0.0-beta

    DataVec ClassPathResource has been deprecated; use nd4j-common version instead Link


Arbiter: New Features

    Added LayerSpace for OCNN (one-class neural network)

Arbiter: Fixes

    Fixed timestamp issue that could cause incorrect rendering of first model's results in UI Link
    Execution now waits for last model(s) to complete before returning when a termination condition is hit Link
    As per DL4J etc: use of Reflections library has been removed entirely from Arbiter Link
    Remove use of Eclipse Collections library due to issues with Android compilation Link
    Improved cleanup of completed models to reduce maximum memory requirements for training Link