1.0.0-beta5
Last updated
Was this helpful?
Last updated
Was this helpful?
Added model server - remote inference of SameDiff and DL4J models using JSON or (optionally) binary serialization
Server: See
Client: See
Tests/examples: See and
Added Scala 2.12 support, dropped Scala 2.10 support. Modules with Scala dependencies are now released with Scala 2.11 and 2.12 versions
Apache Spark 1.x support dropped (now only Spark 2.x is supported). Note: Spark version suffix dropped: For upgrading: 1.0.0-beta4_spark2 -> 1.0.0-beta5
Added FastText support to deeplearning4j-nlp
CUDA support for all ND4J/SameDiff Operations
In 1.0.0-beta4, some operations were CPU only. Now, all operations have full CUDA support
Added support for new data types in ND4J (and DL4J/SameDiff): BFLOAT16, UINT16, UINT32, UINT64
ND4J: Implicit broadcasting support added to INDArray (already present in SameDiff - for example shape [3,1]+[3,2]=[3,2])
CUDA 9.2, 10.0 and 10.1-Update2 still supported
NOTE: For CUDA 10.1, CUDA 10.1 update 2 is recommended. CUDA 10.1 and 10.1 Update 1 will still run, but rare internal cuBLAS issues may be encountered in heavily multi-threaded code on some systems
Dependency upgrades: Jackson (2.5.1 to 2.9.9/2.9.9.3), Commons Compress (1.16.1 to 1.18), Play Framework (2.4.8 to 2.7.3), Guava: (20.0 to 28.0-jre, and shaded to avoid dependency clashes)
CUDA: now host (RAM) buffers are only allocated when required (previously: host buffers were always allocated), in addition to device (GPU) buffer
DL4J AsyncDataSetIterator and AsyncMultiDataSetIterator moved to ND4J, use org.nd4j.linalg.dataset.Async(Multi)DataSetIterator instead
Apache Spark 1.x support dropped (now only Spark 2.x is supported). Note: Spark version suffix dropped: For upgrading, change versions as follows: 1.0.0-beta4_spark2 -> 1.0.0-beta5
Scala 2.10 dropped, Scala 2.12 added (for modules with Scala dependencies)
Some layers (such as LSTM) may run slower on 1.0.0-beta5 than 1.0.0-beta4 on CUDA when not using cuDNN, due to added synchronization. This synchronization will be removed in the next release after 1.0.0-beta5
CUDA 10.1: Rare internal cuBLAS issues may be encountered in heavily multi-threaded code on some systems, when running CUDA 10.1 Update 1 (and maybe 10.1). CUDA 10.1 update 2 is recommended.
CUDA: now host (RAM) buffers are only allocated when required (previously: host buffers were always allocated), in addition to device (GPU) buffer
OldAddOp, OldSubOp, etc removed: Replace with AddOp, SubOp, etc
Nd4j.trueScalar and trueVector removed; use Nd4j.scalar and Nd4j.createFromArray methods
INDArray.javaTensorAlongDimension removed; use INDArray.tensorAlongDimension instead
INDArray.lengthLong() removed; use INDArray.length() instead
Added FastText - inference and training, including OOV (out of vocabulary) support ()
Scala 2.12 support added, Scala 2.10 support dropped ()
Added model server (DL4J and SameDiff models, JSON and binary communication) - , , ,
Added saved model format validation utilities - DL4JModelValidator, DL4JKerasModelValidator ()
Added LabelLastTimeStepPreProcessor ()
BertIterator: added option to prepend token to the output (such as [cls] expected by some models) ()
Added trace level logging to MultiLayerNetwork and ComputationGraph assist with debugging certain issues ()
Upsampling3D: Added NDHWC support ()
MergeVertex now supports broadcasting ()
LSTM and Dropout will now fall back on built-in implementations if an exception is encountered from cuDNN (same as Subsampling/ConvolutionLayer) ()
Improved JavaDoc and cleanup up API for WordVectorSerializer (, )
Updated deeplearning4j-ui theme ()
Fixed an issue with MergeVertex and CNN3D activations ()
Fixed typo in Yolo2OutputLayer builder/configuration method name ()
Improved ComputationGraph builder InputType validation ()
Removed dl4j-spark-ml module until it can be properly maintained ()
Fixed an issue with BertWordPieceTokenizerFactory and bad character encoding ()
Fixed an issue with LearnedSelfAttentionLayer and variable minibatch size (, )
Fixed issue with SharedTrainingMaster controller address when set from environment variable ()
Fixed issue with SameDiffOutputLayer initialization under some circumstances ()
https is now used by default for data and zoo model downloads (, )
Fixed an issue where UI WebJars dependencies would check for updates on every single build (, )
Fixed issue where Upsampling layer memory report could produce an OOM exception ()
Improved UX/validation for RecordReaderDataSetIterator ()
Fixed an issue where EmbeddingSequenceLayer would not check mask array datatype ()
Improved validation when initializing networks with a non rank-2 (shape [1, numParams]) array ()
Fixed a DataType issue for BertIterator ()
Fixed Word2Vec model backward compatibilty (beta3 and earlier models now loadable again)
Fixed issue where some Keras import models could fail with Could not read abnormally long HDF5 attribute ()
Added validation for RnnOutputLayer - feature/label array lengths ()
Fixed an issue where SameDiffOutputLayer would not support variable minibatch size ()
Fixed DL4J SameDiff layer mask support ()
DL4J UI: Fixed an issue where tab switching did not work when visualizing saved/stored data (, )
DL4J UI: Fixed a rare UI threading issue ()
Fixed a Keras import issue with JSON format change ()
Fixed a Keras import issue where updater learning rate schedule could be imported incorrectly ()
Fixed an issue with CnnSentenceDataSetIterator when using UnknownWordHandling.UseUnknownVector (, )
Fixes and optimizations to DL4J SameDiff layers ()
MultiLayerNetwork/ComputationGraph will now log the original exception if a second exception occurs during workspace closing, instead of swallowing it (inference/fit operation try/finally blocks) ()
Upgraded dependencies: Jackson (2.5.1 to 2.9.9/2.9.9.3), Commons Compress (1.16.1 to 1.18), Play Framework (2.4.8 to 2.7.3), Guava: (20.0 to 28.0-jre, shaded to avoid dependency clashes) ()
Logging framework can now be configured for DL4J UI (due to Play framework dependency upgrade) ()
Reduced amount of garbage produced by MnistDataFetcher (impacts MNIST and EMNIST DataSetIterators) ()
Activation function backpropagation has been optimized for many activation functions (, )
Saved models with custom layers from 1.0.0-alpha and before can no longer be loaded. Workaround: load in 1.0.0-beta4, and re-save the model (). Models without custom layers can still be loaded back to 0.5.0
dl4j-spark_2.11 and _2.12 dependencies incorrectly pull in datavec-spark_2.11/2.12 version 1.0.0-SNAPSHOT. Workaround: control version using dependency management as per or
Added new data types: BFLOAT16, UINT16, UINT32, UINT64 ()
CUDA support for all operations without CUDA implementations (, , , , )
Added model server (DL4J and SameDiff models, JSON and binary communication) - , , ,
Added support for empty arrays with zeros in shape, for compatibility with TensorFlow import ()
Improved SameDiff training API - added "in line" test set evaluation, returning History object with loss curve, etc ()
Added saved model format validation utilities - Nd4jValidator, Nd4jCommonValidator ()
Added SameDiff ScoreListener (equivalent to DL4J ScoreIterationListener/PerformanceListener) (, )
Added SameDiff.convertDataTypes method, for variable dtype conversion ()
Added crop and resize op ()
DL4J AsyncDataSetIterator and AsyncMultiDataSetIterator moved to ND4J
Added basic/MVP SameDiff UI listener ()
Added SameDiff CheckpointListener (, )
Added SameDiff name scopes ()
SameDiff: Updater state and training configuration is now written to FlatBuffers format ()
Added c++ benchmark suite callable from Java - call using Nd4j.getExecutioner().runLightBenchmarkSuit() and Nd4j.getExecutioner().runFullBenchmarkSuit() ()
Added SameDiff.save/load methods with InputStream/OutputStream arguments (, )
Added axis configuraiton for evaluation instances (Evaluation, RegressionEvaluation, ROC, etc - getAxis and setAxis methods) to allow different data formats (NCHW vs. NHWC for CNNs, for example) ()
SameDiff: Added support to convert constants to placeholders, via SDVariable.convertToConstant() method ()
SameDiff: Added GradCheckUtil.checkActivationGradients method to check activation gradients for SameDiff instance (not just parameter gradients as in existing gradient check methods) ()
Added CheckNumerics op ()
Added FakeQuantWithMinMaxArgs and FakeQuantWithMinMaxVars ops ()
Added INDArray reduction methods with "keep dimensions" option - for example, INDArray.mean(boloean, int... dimension) ()
Added Nd4j SystemInfo class - SystemInfo.getSystemInfo, .writeSystemInfo(File) to aid with debugging issues (, )
Added INDArray.toString(NDArrayStrings options), toStringFull() and toString overloads for easier control of array printing ()
Added HashCode op, INDArray.hashCode() ()
SameDiff: added whileLoop, ifCond methods for loops/conditional ops ()
Cleaned up some infrequently used Nd4j methods (, , , )
Added bitwise integer operations: left/right bit shift, left/right cyclical bit shift, bitwise Hamming distance (, , , , )
deeplearning4j-nlp: renamed AggregatingSentencePreProcessor to sentencePreProcessor method ()
Upgraded (and shaded) Protobuf version - 3.5.1 to 3.8.0 ()
Switched to c=style error handling for libnd4j native operations ()
Renamed FlatBuffers enum org.nd4j.graph.DataType to org.nd4j.graph.DType to avoid users importing incorrect type when using Nd4j methods (, )
Added SameDiff.bitwise namespace for bitwise ops (, )
Updated to JavaCPP/JavaCV 1.5.1-1 ()
SameDiff: Placeholders must now only be provided if required to calculate the requested variables ()
SameDiff: Fixed an issue with duplicate variable name validation ()
SameDiff: Fixed an issue with SDVariable.getArr for scalars ()
Added delayed mode to DeviceLocalNDArray (don't replicate to device until needed) ()
ND4J: Fixed an issue with writing 0d (scalar) NDArrays in numpy .npy format ()
Fixed an issue with Pad operation for some constant cases ()
Fixed some issues with strided_slice operation (, , )
SameDiff: Fixed issue with DataType inference for some ops using ND4J default datatype ()
INDArray.castTo(DataType) is now a no-op when array is already the correct type ()
SameDiff: Fixed an issue with training mixed precision networks ()
Fixed an issue where Evaluation class was incorrectly reporting macro-averaged precision for binary case ()
Removed trainableParams config/field from SameDiff TrainingConfig (no longer required) ()
Improvements and cleanup to ND4J Javadoc (, , , )
Fixed an issue with Cholesky Lapack op on CUDA (, )
Fixed an issue where [1,N] and [N,1] arrays were not considered a matrix (rank 2 array) according to INDArray.isMatrix() ()
Fixed RegressionEvaluation for 4D arrays (CNNs / segmentation) (, )
Fixed issue with INDArray.median(int... dimension) ()
Fixed NPE that could occur when executing gather operation backprop ()
Fixed issue with LogSumExp operation Java/C++ mapping ()
Added header validation when reading Numpy .npy files, to ensure file is valid ()
Fixed a possible issue with reading Numpy .npy files on CUDA ()
Fixed an issue when reading Numpy .npy boolean files ()
Various fixes for TensorFlow import ()
Fixed an issue with a small number of Nd4j.create methods not creating arrays corresponding to the java primitive ()
Improved shape validation for some Nd4j.create methods ()
Cleaned up unmaintained Nd4j.createSparse methods ()
Fixed a CUDA issue for CUDA GPUs with CC 3.0 ()
Fixed some possible integer overflows in c++ code ()
Removed deprecated methods: Nd4j.trueScalar and Nd4j.trueVector (, )
Fixed an issue where some JVMs could warn about "Illegal reflective access" due to a (now removed) SameDiff dependency ()
SDVariable now no longer extends DifferentialFunction ()
Moved numerous operation calculateOutputShape instances from Java to C++ ()
Fixed an issue where maxpool2d_bp could throw an exception when NaN values are present ()
Fixed an issue with concatenation of empty shapes (with zeros) ()
Removed INDArray.javaTensorAlongDimension ()
LayerNorm operation now properly supports axis arg, NCHW format data ()
libnd4j: cuBLAS hgemm (FP16 gemm) wil only be called for devices with compute capability >= 5.3 due to cuBLAS limitations ()
Nd4j.readNumpy optimized ()
Added configurable alpha parameter to ELU and lrelu_bp operations in c++ ()
Cleaned up SameDiff SDCNN/SDRNN (SameDiff.cnn, .rnn) API/methods (, )
nd4j-native on some OSX systems can fail with Symbol not found: ___emutls_get_address - See
SBT 1.3.0 can fail with an Illegal character in path error; SBT 1.2.8 is OK. This is an SBT issue, not an ND4J issue. See for details
ImageRecordReader: Support for 16-bit TIFF added ()
Added SequenceTrimToLengthTransform ()
Fixed an issue with AnalyzeSpark and String columns ()
Fixed an issue with URL scheme detection in NumberedFileInputScheme ()
Fixed an issue with RandomPathFilter sampling being biased (, )
API cleanup and refactoring (, , , )
Fixed issue with compression for HistoryProcessor ()
Updated EvaluationScoreFunction to use ND4J Evaluation class metrics ()
Fixed incorrect search size in GridSearchCandidateGenerator ()
The Jackson version upgrade necessitated a change to how generic object serialization was performed; Arbiter JSON data stored in 1.0.0-beta4 or earlier format may not be readable in 1.0.0-beta5 ()
Added full data type support to ND4S as per ND4J ()
Added syntactic sugar for SameDiff (implicits, operator overloads) ()