• Workspaces feature added (faster training performance + less memory) Link
  • SharedTrainingMaster added for Spark network training (improved performance) Link 1, Link 2
  • ParallelInference added - wrapper that server inference requests using internal batching and queues Link
  • ParallelWrapper now able to work with gradients sharing, in addition to existing parameters averaging mode Link
  • VPTree performance significantly improved
  • CacheMode network configuration option added - improved CNN and LSTM performance at the expense of additional memory use Link
  • LSTM layer added, with CuDNN support Link (Note that the existing GravesLSTM implementation does not support CuDNN)
  • New native model zoo with pretrained ImageNet, MNIST, and VGG-Face weights Link
  • Convolution performance improvements, including activation caching
  • Custom/user defined updaters are now supported Link
  • Evaluation improvements
    • EvaluationBinary, ROCBinary classes added: for evaluation of binary multi-class networks (sigmoid + xent output layers) Link
    • Evaluation and others now have G-Measure and Matthews Correlation Coefficient support; also macro + micro-averaging support for Evaluation class metrics Link
    • ComputationGraph and SparkComputationGraph evaluation convenience methods added (evaluateROC, etc)
    • ROC and ROCMultiClass support exact calculation (previous: thresholded calculation was used) Link
    • ROC classes now support area under precision-recall curve calculation; getting precision/recall/confusion matrix at specified thresholds (via PrecisionRecallCurve class) Link
    • RegressionEvaluation, ROCBinary etc now support per-output masking (in addition to per-example/per-time-step masking)
    • EvaluationCalibration added (residual plots, reliability diagrams, histogram of probabilities) Link 1 Link 2
    • Evaluation and EvaluationBinary: now supports custom classification threshold or cost array Link
  • Optimizations: updaters, bias calculation
  • Network memory estimation functionality added. Memory requirements can be estimated from configuration without instantiating networks Link 1 Link 2
  • New loss functions:
    • Mixture density loss function Link
    • F-Measure loss function Link
  • Workspaces feature added Link
  • Native parallel sort was added
  • New ops added: SELU/SELUDerivative, TAD-based comparisons, percentile/median, Reverse, Tan/TanDerivative, SinH, CosH, Entropy, ShannonEntropy, LogEntropy, AbsoluteMin/AbsoluteMax/AbsoluteSum, Atan2
  • New distance functions added: CosineDistance, HammingDistance, JaccardDistance
  • MapFileRecordReader and MapFileSequenceRecordReader added Link 1 Link 2
  • Spark: Utilities to save and load JavaRDD<List<Writable>> and JavaRDD<List<List<Writable>> data to Hadoop MapFile and SequenceFile formats Link
  • TransformProcess and Transforms now support NDArrayWritables and NDArrayWritable columns
  • Multiple new Transform classes
  • Arbiter UI: Link
    • UI now uses Play framework, integrates with DL4J UI (replaces Dropwizard backend). Dependency issues/clashing versions fixed.
    • Supports DL4J StatsStorage and StatsStorageRouter mechanisms (FileStatsStorage, Remote UI via RemoveUIStatsStorageRouter)
    • General UI improvements (additional information, formatting fixes)
Last modified 1yr ago