Deeplearning4j
EN 1.0.0-M2.1
Community ForumND4J JavadocDL4J Javadoc
Search
⌃K
Links

1.0.0-beta6

Highlights - 1.0.0-beta6 Release

  • Added support for CUDA 10.2. 1.0.0-beta6 released with CUDA 9.2, 10.0, 10.1 and 10.2 support
  • SameDiff optimizations - memory use for inference and training significantly reduced, with some performance improvements also
  • Deeplearning4j UI - Play framework replaced with Vertx; deeplearning4j-ui dependency now no longer has Scala dependency or Scala version suffix Link​
    • Note: No API changes, only artifact ID change: replace deeplearning4j-ui_2.1x with deeplearning4j-ui
  • ND4j namespace operation methods: operations are available through the Nd4j.math, Nd4j.random, Nd4j.bitwise, Nd4j.nn (neural network), for example Nd4j.math.abs(INDArray), Nd4j.random.logNormal etc Link.
    • Note that additional ND4J namespaces API will have additions (new namespaces and methods), and may have some API changes, in the next release
  • OpenMP replaced with thread pool c++ parallelism framework; enabled c++ parallelism for platforms without C++-level threading for operations

Deeplearning4J

Deeplearning4J: Features and Enhancements

  • DNNL (MKL-DNN) upgraded to version 1.1
  • Added causal convolution mode for Convolution1D layer (ConvolutionMode.Causal) and added causal conv1d support for Keras import Link​
  • Keras import now supports scaled identity weight initialization Link​
  • Added Mish activation function Link, Link​
  • BertIterator now has a BertIterator.featurizeSentences(List<String>) method for inference Link, Link​
  • BertIterator now supports sentence pairs for supervised training Link​
  • Added Sparse multi-class cross entropy for both Deeplearning4j and Keras import Link, Link​
  • Deeplearning4j UI: migrated from Play to Vertx for web serving backend, also removing dependency on Scala libraries; no API changes, only artifact ID change - replace deeplearning4j-ui_2.1x with deeplearning4j-ui Link, Link​
  • Added TimeDistributed wrapper layer Link​

Deeplearning4J: Bug Fixes and Optimizations

  • KDTree implementation optimized Link​
  • Deeplearning4j zoo models and datasets hosting location updated Link​
  • Fixed nIn validation for Deconv2D layer Link​
  • Fixed an issue with incorrect Deconvolution2d results for Keras import models Link​
  • Added DNNL/MKLDNN support for batch normalization layer Link, Link​
  • Fixed various integer casts to avoid overflows for very large arrays (with dimensions or length > Integer.MAX_VALUE) Link​
  • Fixed an issue with UNet non-pretrained model architecture (last layer kernel size) Link​
  • Deeplearning4j SameDiff layers now use DL4J workspaces for better performance and reduced memory consumption Link​
  • Updated broken links in afew error messages Link​
  • Cleaned up a few unused dependencies in various modules Link​
  • Cleaned up duplicate SamplingDataSetIterator class Link​
  • Fixed an issue where ComputationGraph instances with a single input going into multiple embedding layers could throw a NPE Link​
  • Fixed an issue where loss function weights were not automatically cast to network datatype, resulting in an exception if not already correct type Link​
  • Shaded Jackson version upgraded from 2.9.9/2.9.9.3 to 2.10.1 Link​
  • Fixed an issue with KNN where getMostPopulatedClusters actually returned the least populated clusters Link​

Deeplearning4j: Transition Guide, 1.0.0-beta5 to 1.0.0-beta6

  • Deeplearning4j UI artifact ID has changed: deeplearning4j-ui_2.1x (beta5 and earlier) with deeplearning4j-ui

ND4J and SameDiff

ND4J/SameDiff: Features and Enhancements

  • Added suport for CUDA 10.2 Link​
  • DNNL (MKL-DNN) upgraded to version 1.1 Link​
  • Added ND4j namespaces to match SameDiff: Nd4j.math, Nd4j.random, Nd4j.bitwise, Nd4j.nn (neural network) Link​
  • Added SameDiff.calculateGradientsAndOutputs method Link Link​
  • Additional SameDiff single batch .output method overloads for DataSet/MultiDataSet added Link​
  • TensorFlow import ops coverage enhanced (significant number of additional ops supported) Link, Link, Link, Link, Link​
  • PRelu op added Link​
  • adjust_contrast, igamma and igammac ops added Link​
  • ND4J/SameDiff: BitCast, CompareAndBitpack, DivideNoNan, DrawBoundingBoxes, FakeQuantWithMinMaxVarsPerChannel ops added Link​
  • non_max_suppression_overlaps op added Link​
  • ImagePreProcessingScaler now supports segmentation use cases Link​
  • concat operation now supports the concatenation axis being specified via the last input array Link​
  • Added Gamma and Poisson RNG distributions Link​
  • SameDiff’s use of DeviceLocal for variables/constants etc is now configurable Link​
  • Uniform distribution op now supports random integer generation, not just random floating point generation Link​
  • SameDiff: Added simple OpBenchmarkListener for benchmarking purposes Link​
  • Added the ability to disable platform helpers (DNNL/MKLDNN etc) via Nd4jCPU.Environment.getInstance().allowHelpers(false); and Nd4jCuda.Environment.getInstance().allowHelpers(false); Link​
  • Added draw_bounding_boxes operation Link​
  • Added resize_bicubic operation Link​
  • Added causal padding mode to conv1d operation Link​
  • DNNL (MKLDNN) is included and enabled by default for non-AVX builds Link​
  • Added SameDiff ArraySavingListener for debugging purposes Link​

ND4J/SameDiff: Bug Fixes and Optimizations

  • OpenMP replaced with ThreadPool abstraction, enables parallelism for platforms without OpenMP support Link​
  • SameDiff memory management overheauled for (in some cases significantlny) reduced memory consumption and improved performance Link, Link​
  • Switched to Clang instead of gcc for OSX compilation to avoid compiler-related issues Link​
  • Removed SameDiff.outputs() “best guess” output inference due to being unreliable, in favor of explicit SameDiff.setOutputs(String...) call Link​
  • Fixed an issue with Nd4j.hstack on 1D arrays Link​
  • SameDiff no longer allows empty arrays for variables Link​
  • Fixed an issue with Nadam updater LR schedules not being cloned Link​
  • Cleaned up IActivation interface Link​
  • Added new LSTM op implementation with DNNL/MKLDNN support (forward pass only so far) Link​
  • SameDiff API cleaned up; deprecated methods removed Link​
  • Switched SameDiff variable initialization to non-lazy, to avoid unexpected behaviour when mixing execution and ND4J RNG seed setting Link​
  • SameDiff.zero and .one methods now create constants, not vairables Link​
  • Moved CUDA build version and device logging to Java logging, from c++ stdout to enable disabling logging (via ND4J config or slf4j config) Link​
  • Added DNNL/MKLDNN support for batch normalization Link​
  • SameDiff: Fixed an issue where listeners weren’t being called for gradient calculation Link​
  • Added DNNL/MKLDNN support for deconv2d/3d operations Link​
  • Fixed an issue with biasadd_bp operation and NHWC data format Link​
  • Fixed an issue with certain strided slice backprop configurations Link, Link​
  • Fixed an issue with LogSumExp reduction operation backprop for along dimension case Link, Link​
  • INDArray.toString() now has correct brackets for rank 1+ scalars to avoid ambiguity Link​
  • Fixed an issue where some ND4J methods could fail when the library is compiled on Java 9+ but run on Java 8 Link​
  • Fixed empty array input case for is_strictly_increasing, non_decreasing and non_max_suppression ops Link, Link​
  • Fixed empty input arrays for legacy ops (transform, scalar, pairwise, broadcast) Link​
  • CUDA compute capability 3.0 is supported again Link​
  • Improved performance for Scatter operations (1D case) + index validation Link​
  • Fixed an issue where SameDiff TrainingConfig serialization would fail if evaluation instances are set Link, Link​
  • SameDiff execution will now throw an exception when assertion operations in the graph fail Link​
  • PolyGamma function now returns NaNs when passed double for args requiring integer values Link​
  • Fixed some issues for pad and mirror_pad ops to ensure they conform with Tensorflow for imported networks Link​
  • Updated and fixed some issues for TensorFlow graph runner Link​
  • Improved performance for Reverse operation Link​
  • Removed/cleanup up unused ND4J list functionality Link​
  • Fixed reduce bool operation results (such as any, all, IsInf, etc) for empty array inputs Link​

ND4J: Transition Guide, 1.0.0-beta5 to 1.0.0-beta6

  • SameDiff.outputs() now requires user to call SameDiff.setOutputs(String...) first; previous “best guess” output inference was unreliable Link​
  • SameDiff.zero and .one methods now create constants, not vairables Link​

DataVec

DataVec: Bug Fixes and Optimizations

  • NativeImageLoader now checks for empty input streams and throws an exception instead of crashing Link​
  • NDArrayScalarOpTransform now supports modulus operator Link​

RL4J

RL4J: Features and Enhancements

  • Added AsyncTrainingListener Link​
  • Replaced multiple uses of java.util.Random with ND4J Random Link​
  • Added Observable and LegacyMDPWrapper Link​

RL4J: Bug Fixes and Optimizations

  • Refactored RL4J video recording to separate VideoRecorder class Link​
  • Fixed an issue with target for DQN Link, Link​
  • Refactoring for DQN and double DQN for improved maintainability Link​
  • Internal refactoring and various bug fixes Link​

PyDataVec

PyDataVec Features and Enhancements

  • PyDataVec TransformProcess now supports non-inplace operations Link​

PyDataVec Bug Fixes and Optimizations

  • Fixed various issues with PyDataVec Link​
  • Fixed an issue with data locality that could cause incorrect results under some circumstances when running on CUDA Link​
Previous
1.0.0-beta7
Next
1.0.0-beta5
Last modified 9mo ago