1.0.0-beta7
Last updated
Was this helpful?
Last updated
Was this helpful?
Read the announcement at for the highlights of this release.
Added Keras model import support for tf.keras models ,
Full inference and training support is available for ops/layers in the tf.keras namespace; inference only for general Tensorflow operations outside of the tf.keras namespace
Note also improvements to Keras import for reshape, permute, etc operations due to NHWC and NWC support in DL4J
DL4J now supports NHWC (channels last) data format for all CNN 2D layers, in addition to NCHW
DL4J now supports NWC (channels last - [minibatch, sequence_length, size]) for all RNN and CNN 1D layers, in addition to NCW
Added Deconvolution3D layer
Keras import: added ReLU, ELU and Softmax advanced activation layers and Swish activation function
Added DL4J SameDiffLoss class (for easily-defined DL4J ILossFunction's via SameDiff)
Useful exceptions are now thrown when attempting to perform unsupported operations on FastText
Added MultiLayerNetwork.evaluate(MultiDataSetIterator) and .evaluateRegression(MultiDataSetIterator) methods ,
Added new Image operations namespace operations:
Added new Random operations namespace operations:
Added new Math namespace operations:
Added new NN namespace operations:
Added new CNN namespace operations:
Added new linalg operations namespace
Added new RNN operation namespace operations:
Mapped operations for Tensorflow import:
Improved performance for bias_add operation
Updaters (Adam, AdaGrad, etc) optimized via C++ operations (significant training performance boost) for DL4J and SameDiff ,
Some packages relocated to avoid split packages (that can be a problem for OSGi and Java 9 modules)
Note: this is a breaking change for some class packages/imports. See for details on exact package changes
Deeplearning4j UI: Webjars versions locked down using dependency management to avoid check on each build
Added MKLDNN (DNNL/OneDNN) support for depthwise_conv2d operation for DL4J and SameDiff
Refactored/merged modules dl4j-perf and dl4j-util into deeplearning4j-core
Fixed an issue with BertWordPieceTokenizer - potential StackOverflowError with certain inputs
Fixed an issue with GlobalPooling layer with masks of different datatype to the activations datatype
Fixed an issue with DL4JModelValidator for ComputationGraph
Fixed an issue where SameDiff layers in DL4J could throw an exception when used with transfer learning
Weight initialization for EmbeddingLayer and EmbeddingSequenceLayer now no longer depend on the vocabulary size (only the vector size)
Fixed an issue with Keras import with bidirectional layers + preprocessors
DL4J UI: added redirect from /train to /train/overview
Fixed an issue where RecordReaderDataSetIterator builder collectMetaData configuration was not being applied
Fixed an issue where MultiLayerNetwork evaluation was not passing metadata to the IEvaluation instances during evaluation ,
Fixed an issue with Spark training SharedTrainingMaster when training with a ComputationGraph and MultiDataSets
Assorted fixes for edge cases for DL4J Keras import
deelpearning4j-nlp-korean will no longer be released for Scala 2.12 due to required dependency only having Scala 2.11 version avairable
Fix for ConvolutionalIterationListener for ComputationGraph
Fixed an issue where dataset and model zoo downloads could get stuck if the server fails to send any data (now: timeout + retry)
DL4J ModelSerializer no longer writes temporary files when restoring models from InputStream
Fixes issues with UIServer multi session mode, and potential shutdown race condition
Fixed an issue where TfidfVectorizer.vectorize() could throw a NPE when fit from LabelAwareIterator
SameDiff multi-threaded inference enhanced (and fixed) - a single SameDiff instance can now be used for inference safely and efficiently from multiple threads
cuDNN support added to SameDiff (automatically enabled for nd4j-cuda-10.x backend)
Added ND4J namespaces: Nd4j.cnn, Nd4j.rnn, Nd4j.image
rgbToHsv, hsvToRgb
rgbToYiq, yiqToRgb, rgbToYuv, yuvToRgb
imageResize
gamma, poisson, shuffle
clipByAvgNorm, embeddingLookup
mergeMaxIndex
cReLU
upsampling3d
triangular_solve
tri operation
triu operation
lstmLayer (note old lstmLayer method renamed to lstmBlock)
gru
Added new Loss operations namespace - Nd4j.loss
HSVToRGB, RGBToHSV, Igamma, Igammac, RandomGamma, RandomPoisson, RandomPoissonV2, RandomShuffle
Added SameDiff ProfilingListener - writes op performance profiles in Chrome profiler format (load in chrome://tracing/)
Added SameDiff ProfileAnalyzer tool to compare profiles output from ProfilingListener (or Tensorflow)
SameDiff listener API: added frame and iteration information for listener methods
Added (non-backend-specific) method of accessing Nd4j environment: Nd4j.getEnvironment() method (environment info and low-level configuration options)
Improved memory limits/configuration support for libnd4j (c++)
Added pairwise (broadcastable) power backprop operation
Updated JavaCPP presets MKL version to 2020.0 from 2019.5
Added DynamicCustomOp dargs - datatype arguments
Output datatype configuration for Range op , SequenceOp , ConfusionMatrix
Added tensormmul_bp op
OpenBLAS version upgraded to 0.3.8
libnd4j (c++ codebase underlying DL4J, ND4J and SameDiff) refactored to be more easily embeddable in other C++ projects
ImagePreProcessingScaler now supports preprocessing of labels (for segmentation)
Additional datatypes now supported for nd4j-tensorflow TensorflowConversion
SameDiff operation namespaces (sd.math, sd.image, etc) are now code generated to ensure SameDiff and ND4J namespaces are identical (all operations included, same API)
Added ND4J ArchiveUtils.unzipFileTo(String, String, boolean logFiles) overload to enable/disable extracted file path logging
Added weight format configuration for following operations: conv1D, conv2D, conv3D, deconv2d, deconv3d, depthwiseConv2d, pointwiseConv2d, sconv2d
Added backprop operation implementations for mergemax, mergeadd, mergeavg operations
MKL version upgraded to 2020.0 2020.1; OpenCV upgraded from 4.2.0 to 4.3.0
SameDiff: DifferentialFunctionFactory class removed in favor of namespace methods (sd.math, sd.linalg, etc)
Added lstmLayer_bp operation
Added gru_bp operation
linspace operation can now use both targs and arrays for start/end/size arguments
Assorted dependency updates - OpenBLAS (0.3.9), OpenCV (4.3.0), Leptonica (1.79.0)
Upgraded assorted dependency versions: javax.activation:activation (1.1 -> 1.1.1), stream analytics (2.7.0->2.9.8), Apache Spark (2.4.3->2.4.5), Jackson databind (2.10.1 -> 2.10.3), Vertx (3.8.3 -> 3.9.0)
Added nd4j-common-tests ResourceUtils.listClassPathfiles method
Updaters (Adam, AdaGrad, etc) optimized via C++ operations (significant training performance boost) for DL4J and SameDiff ,
SameDiff - added CuDNN support
Some packages relocated to avoid split packages (that can be a problem for OSGi and Java 9 modules)
Note: this is a breaking change for some class packages/imports. See for details on exact package changes
Fixed some issues with Tensorflow import of FusedBatchNorm operation
Fixed an issue where the Roll operation did not match Tensorflow operation
Fixed an issue where ArchiveUtils could fail to create the top level destination directory when it does not exist
Fixed an issue where resize_bicubic operation did not match Tensorflow for some configuration values
Pad operation now supports long/int64 values for padding array
Fixed an issue where hashcode operation shape function wasn't always returning int64/long dtype
Fixed an issue with reshape operation on empty arrays with -1s
Improved performance on CUDA for concat operation and CPU/GPU
On CPU for NHWC case
Generally
On CUDA for 2D case
Added MKLDNN (DNNL/OneDNN) support for depthwise_conv2d operation for DL4J and SameDiff
Fixed a small SameDiff execution issue for switch operation where the predicate is a constant
Fixed an issue with batchnorm operation when input arrays have unusual strides
Merged nd4j-buffer, nd4j-content modules into nd4j-api
Deleted deprecated nd4j-jackson module (remaining functionality available in nd4j-api)
Deleted unused/unmaintained nd4j-camel and nd4j-gson modules
Optimization for legacy random ops
Optimization for broadcast operations , , , ,
Performance optimization for multiple operations: softmax, squeeze, expand_dims, tanh
Optimization for transpose/permute operations
Performance enhancement: MKLDNN matmul used for some mmul operation cases
Optimization for gather operation on CPU
Optimization for stack/unstack operations on CPU
Optimization for split operation (CPU and CUDA)
ND4J initialization no longer logs number of OpenMP BLAS threads for CUDA
Optimization: Fixed issues with auto-vectorization on multple CPU operations
Optimization for reshape operation ,
Fixed an issue where INDArray.hashCode() could cause an exception on some datatypes
Optimization for CPU: MKLDNN is now used for softmax, tanh, softmax_bp and tanh_bp operations , , ,
Fixed random_exponential operation
Improved performance on C++ SameDiff graph execution via reduced array zeroing where safe to do so
Improved C++ indexing implementation impacting CPU performance on some operations
Fixed an issue where Split operation could have incorrect output shapes for empty arrays
Fixed some issues with SameDiff.equals method
Fixed an issue with reshape operation output shape on empty arrays ,
Nd4j.gemm now uses Mmul operation internally to avoid potential threading issues with direct BLAS calls on CUDA
Fixed an edge case issue with percentile operation
Fixed an edge case issue for cusolved (CUDA) in libnd4j
Fixed an issue with error formatting for segment operations for incorrect lengths
Fixed an issue where ND4J workspaces were not guaranteed to be unique
Fixed some operation implementations when operating on views (Batch/Space to Space/Batch/Depth; batchnorm_bp)
Fixed an issue where exponential distribution random number generation operation could produce infinities extremely rarely (~1 in 10^9 values)
Fixed an issue with long file paths for memory mapped workspaces on Windows
Memory for memory mapped workspaces are now deallocated immediately when workspace is destroyed, instead of waiting for GC to free memory
Fall-back to other BLAS implementation for cases where MKLDNN GEMM implementation is slow
Set nd4j-native source/target to Java 7 ,
datavec-python: added zero-copy support for bytes/byte buffers
datavec-python: Python exceptions are now thrown as Java exceptions
datavec-python: Added support for additional NumPy datatypes
datavec-python: Python version upgraded from 3.7.6 to 3.7.7
Deleted not properly maintained modules: datavec-camel, datavec-perf
Fixed missing BOOL datatype support for arrow conversion functionality
Assorted fixes for datavec-python ,
Fixed an issue with LineRecordReader where initialization was performed unnecessarily (adding performance overhead)
Refactoring to decouple configuration and learning methods from their implementations
Added builder patterns for all configuration classes
Fixes an issue with GridSearchCandidateGenerator not working correctly for some cases ,