Deeplearning4j
Community ForumND4J JavadocDL4J Javadoc
EN 1.0.0-M2
EN 1.0.0-M2
  • Deeplearning4j Suite Overview
  • Release Notes
    • 1.0.0-M2
    • 1.0.0-M1.1
    • 1.0.0-M1
    • 1.0.0-beta7
    • 1.0.0-beta6
    • 1.0.0-beta5
    • 1.0.0-beta4
    • 1.0.0-beta3
    • 1.0.0-beta2
    • 1.0.0-beta
    • 1.0.0-alpha
    • 0.9.1
    • 0.9.0
    • 0.8.0
    • 0.7.2
    • 0.7.1
    • 0.7.0
    • 0.6.0
    • 0.5.0
    • 0.4.0
    • 1.00-M2.2
  • Multi-Project
    • Tutorials
      • Beginners
      • Quickstart
    • How To Guides
      • Import in to your favorite IDE
      • Contribute
        • Eclipse Contributors
      • Developer Docs
        • Github Actions/Build Infra
        • Javacpp
        • Release
        • Testing
      • Build From Source
      • Benchmark
      • Beginners
    • Reference
      • Examples Tour
    • Explanation
      • The core workflow
      • Configuration
        • Backends
          • Performance Issues
          • CPU
          • Cudnn
        • Memory
          • Workspaces
      • Build Tools
      • Snapshots
      • Maven
  • Deeplearning4j
    • Tutorials
      • Quick Start
      • Language Processing
        • Doc2Vec
        • Sentence Iterator
        • Tokenization
        • Vocabulary Cache
    • How To Guides
      • Custom Layers
      • Keras Import
        • Functional Models
        • Sequential Models
        • Custom Layers
        • Keras Import API Overview
          • Advanced Activations
          • Convolutional Layers
          • Core Layers
          • Embedding Layers
          • Local Layers
          • Noise Layers
          • Normalization Layers
          • Pooling Layers
          • Recurrent Layers
          • Wrapper Layers
        • Supported Features Overview
          • Activations
          • Constraints
          • Initializers
          • Losses
          • Optimizers
          • Regularizers
      • Tuning and Training
        • Visualization
        • Troubleshooting Training
        • Early Stopping
        • Evaluation
        • Transfer Learning
    • Reference
      • Model Zoo
        • Zoo Models
      • Activations
      • Auto Encoders
      • Computation Graph
      • Convolutional Layers
      • DataSet Iterators
      • Layers
      • Model Listeners
      • Saving and Loading Models
      • Multi Layer Network
      • Recurrent Layers
      • Updaters/Optimizers
      • Vertices
      • Word2vec/Glove/Doc2Vec
    • Explanation
  • datavec
    • Tutorials
      • Overview
    • How To Guides
    • Reference
      • Analysis
      • Conditions
      • Executors
      • Filters
      • Normalization
      • Operations
      • Transforms
      • Readers
      • Records
      • Reductions
      • Schemas
      • Serialization
      • Visualization
    • Explanation
  • Nd4j
    • Tutorials
      • Quickstart
    • How To Guides
      • Other Framework Interop
        • Tensorflow
        • TVM
        • Onnx
      • Matrix Manipulation
      • Element wise Operations
      • Basics
    • Reference
      • Op Descriptor Format
      • Tensor
      • Syntax
    • Explanation
  • Samediff
    • Tutorials
      • Quickstart
    • How To Guides
      • Importing Tensorflow
      • Adding Operations
        • codegen
    • Reference
      • Operation Namespaces
        • Base Operations
        • Bitwise
        • CNN
        • Image
        • LinAlg
        • Loss
        • Math
        • NN
        • Random
        • RNN
      • Variables
    • Explanation
      • Model Import Framework
  • Libnd4j
    • How To Guides
      • Building on Windows
      • Building for raspberry pi or Jetson Nano
      • Building on ios
      • How to Add Operations
      • How to Setup CLion
    • Reference
      • Understanding graph execution
      • Overview of working with libnd4j
      • Helpers Overview (CUDNN, OneDNN,Armcompute)
    • Explanation
  • Python4j
    • Tutorials
      • Quickstart
    • How To Guides
      • Write Python Script
    • Reference
      • Python Types
      • Python Path
      • Garbage Collection
      • Python Script Execution
    • Explanation
  • Spark
    • Tutorials
      • DL4J on Spark Quickstart
    • How To Guides
      • How To
      • Data How To
    • Reference
      • Parameter Server
      • Technical Reference
    • Explanation
      • Spark API Reference
  • codegen
Powered by GitBook
On this page
  • Operation classes
  • gru
  • gruCell
  • lstmCell
  • lstmLayer
  • lstmblock
  • sru
  • sruCell
  • Configuration Classes
  • LSTMConfiguration
  • LSTMLayerConfig
  • GRUWeights
  • SRUWeights
  • LSTMWeights
  • LSTMLayerWeights

Was this helpful?

Export as PDF
  1. Samediff
  2. Reference
  3. Operation Namespaces

RNN

Operation classes

gru

INDArray gru(INDArray x, INDArray hLast, INDArray Wx, INDArray Wh, INDArray biases)

SDVariable gru(SDVariable x, SDVariable hLast, SDVariable Wx, SDVariable Wh, SDVariable biases)
SDVariable gru(String name, SDVariable x, SDVariable hLast, SDVariable Wx, SDVariable Wh, SDVariable biases)

The GRU operation. Gated Recurrent Unit - Cho et al. 2014.

  • x (NUMERIC) - input [time, bS, nIn]

  • hLast (NUMERIC) - initial cell output (at time step = 0) [bS, nOut]

  • Wx (NUMERIC) - input-to-hidden weights, [nIn, 3*nOut]

  • Wh (NUMERIC) - hidden-to-hidden weights, [nOut, 3*nOut]

  • biases (NUMERIC) - biases, [3*nOut]

gruCell

INDArray[] gruCell(INDArray x, INDArray hLast, GRUWeights gRUWeights)

SDVariable[] gruCell(SDVariable x, SDVariable hLast, GRUWeights gRUWeights)
SDVariable[] gruCell(String name, SDVariable x, SDVariable hLast, GRUWeights gRUWeights)

The GRU cell. Does a single time step operation

  • x (NUMERIC) - Input, with shape [batchSize, inSize]

  • hLast (NUMERIC) - Output of the previous cell/time step, with shape [batchSize, numUnits]

lstmCell

INDArray[] lstmCell(INDArray x, INDArray cLast, INDArray yLast, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)

SDVariable[] lstmCell(SDVariable x, SDVariable cLast, SDVariable yLast, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)
SDVariable[] lstmCell(String name, SDVariable x, SDVariable cLast, SDVariable yLast, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)

The LSTM cell. Does a single time step operation.

  • x (NUMERIC) - Input, with shape [batchSize, inSize]

  • cLast (NUMERIC) - Previous cell state, with shape [batchSize, numUnits]

  • yLast (NUMERIC) - revious cell output, with shape [batchSize, numUnits]

lstmLayer

INDArray[] lstmLayer(INDArray x, INDArray cLast, INDArray yLast, INDArray maxTSLength, LSTMLayerWeights lSTMLayerWeights, LSTMLayerConfig lSTMLayerConfig)
INDArray[] lstmLayer(INDArray x, LSTMLayerWeights lSTMLayerWeights, LSTMLayerConfig lSTMLayerConfig)

SDVariable[] lstmLayer(SDVariable x, SDVariable cLast, SDVariable yLast, SDVariable maxTSLength, LSTMLayerWeights lSTMLayerWeights, LSTMLayerConfig lSTMLayerConfig)
SDVariable[] lstmLayer(SDVariable x, LSTMLayerWeights lSTMLayerWeights, LSTMLayerConfig lSTMLayerConfig)
SDVariable[] lstmLayer(String name, SDVariable x, SDVariable cLast, SDVariable yLast, SDVariable maxTSLength, LSTMLayerWeights lSTMLayerWeights, LSTMLayerConfig lSTMLayerConfig)
SDVariable[] lstmLayer(String name, SDVariable x, LSTMLayerWeights lSTMLayerWeights, LSTMLayerConfig lSTMLayerConfig)

Long Short-Term Memory layer - Hochreiter 1997.

SUPPORTS following data formats:

for unidirectional:

TNS: shapes [timeLength, numExamples, inOutSize]

NST: shapes [numExamples, inOutSize, timeLength]

NTS: shapes [numExamples, timeLength, inOutSize]

for bidirectional:

T2NS: shapes [timeLength, 2, numExamples, inOutSize] (for ONNX)

SUPPORTS following direction modes:

FWD: forward

BWD: backward

BIDIR_SUM: bidirectional sum

BIDIR_CONCAT: bidirectional concat

BIDIR_EXTRA_DIM: bidirectional extra output dim (in conjunction with format dataFormat - T2NS)

You may use different gate configurations:

specify gate/cell/out aplha/beta and numbers of activations for gate/cell/out described in activations enum

("RELU","SIGMOID","AFFINE","LEAKY_RELU","THRESHHOLD_RELU","SCALED_TAHN","HARD_SIGMOID","ELU","SOFTSIGN","SOFTPLUS")

Also this layer supports MKLDNN (DNNL) and cuDNN acceleration

  • x (NUMERIC) - Input, with shape dependent on the data format (in config).

  • cLast (NUMERIC) - Previous/initial cell state, with shape [batchSize, numUnits]

  • yLast (NUMERIC) - Previous/initial cell output, with shape [batchSize, numUnits]

  • maxTSLength (NUMERIC) - maxTSLength with shape [batchSize]

lstmblock

INDArray lstmblock(INDArray maxTSLength, INDArray x, INDArray cLast, INDArray yLast, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)
INDArray lstmblock(INDArray x, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)

SDVariable lstmblock(SDVariable maxTSLength, SDVariable x, SDVariable cLast, SDVariable yLast, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)
SDVariable lstmblock(SDVariable x, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)
SDVariable lstmblock(String name, SDVariable maxTSLength, SDVariable x, SDVariable cLast, SDVariable yLast, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)
SDVariable lstmblock(String name, SDVariable x, LSTMWeights lSTMWeights, LSTMConfiguration lSTMConfiguration)

The LSTM block

  • maxTSLength (NUMERIC) -

  • x (NUMERIC) - Input, with shape dependent on the data format (in config).

  • cLast (NUMERIC) - Previous/initial cell state, with shape [batchSize, numUnits]

  • yLast (NUMERIC) - Previous/initial cell output, with shape [batchSize, numUnits]

sru

INDArray sru(INDArray x, INDArray initialC, INDArray mask, SRUWeights sRUWeights)
INDArray sru(INDArray x, INDArray initialC, SRUWeights sRUWeights)

SDVariable sru(SDVariable x, SDVariable initialC, SDVariable mask, SRUWeights sRUWeights)
SDVariable sru(SDVariable x, SDVariable initialC, SRUWeights sRUWeights)
SDVariable sru(String name, SDVariable x, SDVariable initialC, SDVariable mask, SRUWeights sRUWeights)
SDVariable sru(String name, SDVariable x, SDVariable initialC, SRUWeights sRUWeights)

The SRU layer. Does a single time step operation.

  • x (NUMERIC) - Input, with shape [batchSize, inSize]

  • initialC (NUMERIC) - Initial cell state, with shape [batchSize, inSize]

  • mask (NUMERIC) - An optional dropout mask, with shape [batchSize, inSize]

sruCell

INDArray sruCell(INDArray x, INDArray cLast, SRUWeights sRUWeights)

SDVariable sruCell(SDVariable x, SDVariable cLast, SRUWeights sRUWeights)
SDVariable sruCell(String name, SDVariable x, SDVariable cLast, SRUWeights sRUWeights)

The SRU layer. Does a single time step operation.

  • x (NUMERIC) - Input, with shape [batchSize, inSize]

  • cLast (NUMERIC) - Previous cell state, with shape [batchSize, inSize]

Configuration Classes

LSTMConfiguration

  • RnnDataFormat (ENUM) - The data format of the input. Input shape depends on data format (in config):

TNS -> [timeSteps, batchSize, inSize]

NST -> [batchSize, inSize, timeSteps]

NTS -> [batchSize, timeSteps, inSize]

  • peepHole (BOOL) - Whether to provide peephole connections

  • forgetBias (NUMERIC) - The bias added to forget gates in order to reduce the scale of forgetting in the beginning of the training.

  • clippingCellValue (NUMERIC) - The bias added to forget gates in order to reduce the scale of forgetting in the beginning of the training.

LSTMLayerConfig

  • LSTMDataFormat (ENUM) - for unidirectional: TNS: shape [timeLength, numExamples, inOutSize] - sometimes referred to as "time major"

NST: shape [numExamples, inOutSize, timeLength]

NTS: shape [numExamples, timeLength, inOutSize] - TF "time_major=false" layout for bidirectional:

T2NS: 3 = [timeLength, 2, numExamples, inOutSize] (for ONNX)

  • LSTMDirectionMode (ENUM) - direction

FWD: 0 = fwd

BWD: 1 = bwd

BIDIR_SUM: 2 = bidirectional sum

BIDIR_CONCAT: 3 = bidirectional concat

BIDIR_EXTRA_DIM: 4 = bidirectional extra output dim (in conjunction with format dataFormat = 3)

  • gateAct (ENUM) - Activations

  • cellAct (ENUM) - Activations

  • outAct (ENUM) - Activations

  • retFullSequence (BOOL) - indicates whether to return whole time sequence h {h_0, h_1, ... , h_sL-1} - default = true

  • retLastH (BOOL) - indicates whether to return output at last time step only,

    in this case shape would be [bS, nOut] (exact shape depends on dataFormat argument) - default = false

  • retLastC (BOOL) - indicates whether to return cells state at last time step only,

    in this case shape would be [bS, nOut] (exact shape depends on dataFormat argument) - default = false

  • cellClip (NUMERIC) - Cell clipping value, if it = 0 then do not apply clipping - default = 0.0

  • gateAlpha (NUMERIC) - null - default = 0.0

  • gateBeta (NUMERIC) - null - default = 0.0

  • cellAlpha (NUMERIC) - null - default = 0.0

  • cellBeta (NUMERIC) - null - default = 0.0

  • outAlpha (NUMERIC) - null - default = 0.0

  • outBeta (NUMERIC) - null - default = 0.0

GRUWeights

  • ruWeight- null (NUMERIC type)

  • cWeight- null (NUMERIC type)

  • ruBias- null (NUMERIC type)

  • cBias- null (NUMERIC type)

SRUWeights

  • weights- null (NUMERIC type)

  • bias- null (NUMERIC type)

LSTMWeights

  • ruWeight- null (NUMERIC type)

  • inputPeepholeWeights- null (NUMERIC type)

  • forgetPeepholeWeights- null (NUMERIC type)

  • outputPeepholeWeights- null (NUMERIC type)

  • bias- null (NUMERIC type)

LSTMLayerWeights

  • inputWeights- input weights Wx:

    1) shapes [nIn, 4*nOut] for FWD,BWD 2) shapes [2, nIn, 4*nOut] BIDIR_SUM, BIDIR_CONCAT and BIDIR_EXTRA_DIM (NUMERIC type)

  • recurrentWeights- recurrent weights Wr:

    1) shapes [nIn, 4*nOut] for FWD, BWD 2) shapes [2, nIn, 4*nOut] BIDIR_SUM, BIDIR_CONCAT and BIDIR_EXTRA_DIM (NUMERIC type)

  • biases- biases

    1) shapes [4*nOut] for FWD, BWD 2) shapes [2, 4*nOut] for BIDIR_SUM, BIDIR_CONCAT and BIDIR_EXTRA_DIM (NUMERIC type)

  • peepholeWeights- peephole weights Wp:

    1) [3*nOut] when directionMode < 2

    2) [2, 3*nOut] when directionMode >= 2 (NUMERIC type)

PreviousRandomNextVariables

Last updated 3 years ago

Was this helpful?

GRUWeights - see

LSTMWeights - see

LSTMConfiguration - see

LSTMLayerWeights - see

LSTMLayerConfig - see

LSTMWeights - see

LSTMConfiguration - see

SRUWeights - see

SRUWeights - see

Used in these ops:

Used in these ops:

Used in these ops:

Used in these ops:

Used in these ops:

Used in these ops:

GRUWeights
LSTMWeights
LSTMConfiguration
LSTMLayerWeights
LSTMLayerConfig
LSTMWeights
LSTMConfiguration
SRUWeights
SRUWeights
lstmCell
lstmblock
lstmLayer
gruCell
sru
sruCell
lstmCell
lstmblock
lstmLayer