Deeplearning4j
Updater configuration methods such as .momentum(double) and .epsilon(double) have been deprecated. Instead: use .updater(new Nesterovs(0.9))
and .updater(Adam.builder().beta1(0.9).beta2(0.999).build())
etc to configure
DataVec
CsvRecordReader constructors: now uses characters for delimiters, instead of Strings (i.e., ',' instead of ",")
Arbiter
Arbiter UI is now a separate module, with Scala version suffixes: arbiter-ui_2.10
and arbiter-ui_2.11
Added transfer learning API Link
Spark 2.0 support (DL4J and DataVec; see transition notes below)
New ComputationGraph vertices
L2 distance vertex
L2 normalization vertex
Per-output masking is now supported for most loss functions (for per output masking, use a mask array equal in size/shape to the labels array; previous masking functionality was per-example for RNNs)
L1 and L2 regularization can now be configured for biases (via l1Bias and l2Bias configuration options)
Evaluation improvements:
DL4J now has an IEvaluation class (that Evaluation, RegressionEvaluation, etc all implement. Also allows custom evaluation on Spark) Link
Added multi-class (one vs. all) ROC: ROCMultiClass Link
For both MultiLayerNetwork and SparkDl4jMultiLayer: added evaluateRegression, evaluateROC, evaluateROCMultiClass convenience methods
HTML export functionality added for ROC charts Link
TSNE re-added to new UI
Training UI: now usable without an internet connection (no longer relies on externally hosted fonts)
UI: improvements to error handling for ‘no data’ condition
Epsilon configuration now used for Adam and RMSProp updaters
Fix for bidirectional LSTMs + variable-length time series (using masking)
Spark + Kryo: now test serialization + throw exception if misconfigured (instead of logging an error that can be missed)
MultiLayerNetwork now adds default layer names if no name is specified
DataVec:
JSON/YAML support for DataAnalysis, custom Transforms etc
ImageRecordReader refactored to reduce garbage collection load (hence improve performance with large training sets)
Faster quality analysis.
Arbiter: added new layer types to match DL4J
Performance improvement for Word2Vec/ParagraphVectors tokenization & training.
Batched inference introduced for ParagraphVectors
Nd4j improvements
New native operations available for ND4j: firstIndex, lastIndex, remainder, fmod, or, and, xor.
OpProfiler NAN_PANIC & INF_PANIC now also checks result of BLAS calls.
Nd4.getMemoryManager() now provides methods to tweak GC behavior.
Alpha version of parameter server for Word2Vec/ParagraphVectors were introduced for Spark. Please note: It’s not recommended for production use yet.
Performance improvements for CNN inference
Spark versioning schemes: with the addition of Spark 2 support, the versions for Deeplearning4j and DataVec Spark modules has changed
For Spark 1: use <version>0.8.0_spark_1</version>
For Spark 2: use <version>0.8.0_spark_2</version>
Also note: Modules with Spark 2 support are released with Scala 2.11 support only. Spark 1 modules are released with both Scala 2.10 and 2.11 support
UI/CUDA/Linux issue: Link
Dirty shutdown on JVM exit is possible for CUDA backend sometimes: Link
Issues with RBM implementation Link
Keras 1D convolutional and pooling layers cannot be imported yet. Will be supported in forthcoming release.
Keras v2 model configurations cannot be imported yet. Will be supported in forthcoming release.