0.8.0

0.8.0 -> 0.9.0 Transition Notes

Deeplearning4j

• Updater configuration methods such as .momentum(double) and .epsilon(double) have been deprecated. Instead: use .updater(new Nesterovs(0.9)) and .updater(Adam.builder().beta1(0.9).beta2(0.999).build()) etc to configure

DataVec

• CsvRecordReader constructors: now uses characters for delimiters, instead of Strings (i.e., ',' instead of ",")

Arbiter

• Arbiter UI is now a separate module, with Scala version suffixes: arbiter-ui_2.10 and arbiter-ui_2.11

Version 0.8.0

• Added transfer learning API Link

• Spark 2.0 support (DL4J and DataVec; see transition notes below)

• New layers

  • Global pooling (aka "pooling over time"; usable with both RNNs and CNNs) Link

  • Center loss output layer Link

  • 1D Convolution and subsampling layers Link Link2

  • ZeroPaddingLayer Link

• New ComputationGraph vertices

  • L2 distance vertex

  • L2 normalization vertex

• Per-output masking is now supported for most loss functions (for per output masking, use a mask array equal in size/shape to the labels array; previous masking functionality was per-example for RNNs)

• L1 and L2 regularization can now be configured for biases (via l1Bias and l2Bias configuration options)

• Evaluation improvements:

  • DL4J now has an IEvaluation class (that Evaluation, RegressionEvaluation, etc all implement. Also allows custom evaluation on Spark) Link

  • Added multi-class (one vs. all) ROC: ROCMultiClass Link

  • For both MultiLayerNetwork and SparkDl4jMultiLayer: added evaluateRegression, evaluateROC, evaluateROCMultiClass convenience methods

  • HTML export functionality added for ROC charts Link

  • TSNE re-added to new UI

  • Training UI: now usable without an internet connection (no longer relies on externally hosted fonts)

  • UI: improvements to error handling for ‘no data’ condition

• Epsilon configuration now used for Adam and RMSProp updaters

• Fix for bidirectional LSTMs + variable-length time series (using masking)

• Added CnnSentenceDataSetIterator (for use with ‘CNN for Sentence Classification’ architecture) Link Link2

• Spark + Kryo: now test serialization + throw exception if misconfigured (instead of logging an error that can be missed)

• MultiLayerNetwork now adds default layer names if no name is specified

• DataVec:

  • JSON/YAML support for DataAnalysis, custom Transforms etc

  • ImageRecordReader refactored to reduce garbage collection load (hence improve performance with large training sets)

  • Faster quality analysis.

• Arbiter: added new layer types to match DL4J

  • Performance improvement for Word2Vec/ParagraphVectors tokenization & training.

• Batched inference introduced for ParagraphVectors

• Nd4j improvements

  • New native operations available for ND4j: firstIndex, lastIndex, remainder, fmod, or, and, xor.

  • OpProfiler NAN_PANIC & INF_PANIC now also checks result of BLAS calls.

  • Nd4.getMemoryManager() now provides methods to tweak GC behavior.

• Alpha version of parameter server for Word2Vec/ParagraphVectors were introduced for Spark. Please note: It’s not recommended for production use yet.

• Performance improvements for CNN inference

0.7.2 -> 0.8.0 Transition Notes

• Spark versioning schemes: with the addition of Spark 2 support, the versions for Deeplearning4j and DataVec Spark modules has changed

  • For Spark 1: use <version>0.8.0_spark_1</version>

  • For Spark 2: use <version>0.8.0_spark_2</version>

  • Also note: Modules with Spark 2 support are released with Scala 2.11 support only. Spark 1 modules are released with both Scala 2.10 and 2.11 support

0.8.0 Known Issues (At Launch)

• UI/CUDA/Linux issue: Link

• Dirty shutdown on JVM exit is possible for CUDA backend sometimes: Link

• Issues with RBM implementation Link

• Keras 1D convolutional and pooling layers cannot be imported yet. Will be supported in forthcoming release.

• Keras v2 model configurations cannot be imported yet. Will be supported in forthcoming release.