1 of 13

Reference

Operation Namespaces

All operations in ND4J and SameDiff are available in "Operation Namespaces". Each namespace is available on the Nd4j and SameDiff classes with its lowercase name.

For example, if you want to use the absoluteDifference operation it would look like this

// ND4J mode
INDArray output = Nd4j.loss.absoluteDifference(labels, predictions, null);

// SameDiff mode
SDVariable output = SameDiff.loss.absoluteDifference(labels, predictions, null);

Namespaces

Bitwise

and

Bitwise AND operation. Supports broadcasting.

x (INT) - First input array

Image

CropAndResize

Given an input image and some crop boxes, extract out the image subsets and resize them to the specified size.

image (NUMERIC) - Input image, with shape [batch, height, width, channels]
cropBoxes (NUMERIC) - Float32 crop, shape [numBoxes, 4] with values in range 0 to 1
boxIndices (NUMERIC) - Indices: which image (index to dimension 0) the cropBoxes belong to. Rank 1, shape [numBoxes]
cropOutSize (INT) - Output size for the images - int32, rank 1 with values [outHeight, outWidth]
extrapolationValue - Used for extrapolation, when applicable. 0.0 should be used for the default - default = 0.0

adjustContrast

Adjusts contrast of RGB or grayscale images.

in (NUMERIC) - images to adjust. 3D shape or higher
factor - multiplier for adjusting contrast

adjustHue

Adjust hue of RGB image

in (NUMERIC) - image as 3D array
delta - value to add to hue channel

adjustSaturation

Adjust saturation of RGB images

in (NUMERIC) - RGB image as 3D array
factor - factor for saturation

extractImagePatches

Given an input image, extract out image patches (of size kSizes - h x w) and place them in the depth dimension.

image (NUMERIC) - Input image to extract image patches from - shape [batch, height, width, channels]
kSizes - Kernel size - size of the image patches, [height, width] (Size: Exactly(count=2))
strides - Stride in the input dimension for extracting image patches, [stride_height, stride_width] (Size: Exactly(count=2))

hsvToRgb

Converting image from HSV to RGB format

input (NUMERIC) - 3D image

imageResize

Resize images to size using the specified method.

input (NUMERIC) - 4D image [NHWC]
size (INT) - new height and width
preserveAspectRatio - Whether to preserve the aspect ratio. If this is set, then images will be resized to a size that fits in size while preserving the aspect ratio of the original image. Scales up the image if size is bigger than the current size of the image. Defaults to False. - default = false

nonMaxSuppression

Greedily selects a subset of bounding boxes in descending order of score

boxes (NUMERIC) - Might be null. Name for the output variable
scores (NUMERIC) - vector of shape [num_boxes]
maxOutSize - scalar representing the maximum number of boxes to be selected

randomCrop

Randomly crops image

input (NUMERIC) - input array
shape (INT) - shape for crop

rgbToHsv

Converting array from HSV to RGB format

input (NUMERIC) - 3D image

rgbToYiq

Converting array from RGB to YIQ format

input (NUMERIC) - 3D image

rgbToYuv

Converting array from RGB to YUV format

input (NUMERIC) - 3D image

yiqToRgb

Converting image from YIQ to RGB format

input (NUMERIC) - 3D image

yuvToRgb

Converting image from YUV to RGB format

input (NUMERIC) - 3D image

LinAlg

Cholesky

Computes the Cholesky decomposition of one or more square matrices.

input (NUMERIC) - Input tensor with inner-most 2 dimensions forming square matrices

Lstsq

Solver for linear squares problems.

matrix (NUMERIC) - input tensor
rhs (NUMERIC) - input tensor
l2_reguralizer - regularizer

Lu

Computes LU decomposition.

input (NUMERIC) - input tensor

Matmul

Performs matrix mutiplication on input tensors.

a (NUMERIC) - input tensor
b (NUMERIC) - input tensor

MatrixBandPart

Copy a tensor setting outside a central band in each innermost matrix.

input (NUMERIC) - input tensor
minLower - lower diagonal count
maxUpper - upper diagonal count

Qr

Computes the QR decompositions of input matrix.

input (NUMERIC) - input tensor
full - full matrices mode - default = false

Solve

Solver for systems of linear equations.

matrix (NUMERIC) - input tensor
rhs (NUMERIC) - input tensor
adjoint - adjoint mode, defaults to False - default = false

TriangularSolve

Solver for systems of linear questions.

matrix (NUMERIC) - input tensor
rhs (NUMERIC) - input tensor
lower - defines whether innermost matrices in matrix are lower or upper triangular

cross

Computes pairwise cross product.

a (NUMERIC) -
b (NUMERIC) -

diag

Calculates diagonal tensor.

input (NUMERIC) -

diag_part

Calculates diagonal tensor.

input (NUMERIC) -

logdet

Calculates log of determinant.

input (NUMERIC) -

mmul

Matrix multiplication: out = mmul(x,y)

Supports specifying transpose argument to perform operation such as mmul(a^T, b), etc.

x (NUMERIC) - First input variable
y (NUMERIC) - Second input variable
transposeX - Transpose x (first argument) - default = false

svd

Calculates singular value decomposition.

input (NUMERIC) -
fullUV -
computeUV -

tri

An array with ones at and below the given diagonal and zeros elsewhere.

dataType - Data type - default = DataType.FLOAT
row -
column -

triu

Upper triangle of an array. Return a copy of a input tensor with the elements below the k-th diagonal zeroed.

input (NUMERIC) -
diag - - default = 0

Loss

absoluteDifference

Absolute difference loss: sum_i abs( label[i] - predictions[i] )

label (NUMERIC) - Label array
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used
lossReduce - Reduction type for the loss. See LossReduce for more details. Default: LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNT - default = LossReduce.MEAN_BY_NONZERO_WEIGHT_COUNT

cosineDistance

Cosine distance loss: 1 - cosineSimilarity(x,y) or 1 - sum_i label[i] * prediction[i], which is

equivalent to cosine distance when both the predictions and labels are normalized. Note: This loss function assumes that both the predictions and labels are normalized to have unit l2 norm.

If this is not the case, you should normalize them first by dividing by norm2(String, SDVariable, boolean, int...)

along the cosine distance dimension (with keepDims=true).

label (NUMERIC) - Label array
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is use

hingeLoss

Hinge loss: a loss function used for training classifiers.

Implements L = max(0, 1 - t * predictions) where t is the label values after internally converting to {-1,1`

from the user specified {0,1. Note that Labels should be provided with values {0,1.

label (NUMERIC) - Label array. Each value should be 0.0 or 1.0 (internally -1 to 1 is used)
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

huberLoss

Huber loss function, used for robust regression. It is similar both squared error loss and absolute difference loss,

though is less sensitive to outliers than squared error. Huber loss implements:

label (NUMERIC) - Label array
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

l2Loss

L2 loss: 1/2 * sum(x^2)

var (NUMERIC) - Variable to calculate L2 loss of

logLoss

Log loss, i.e., binary cross entropy loss, usually used for binary multi-label classification. Implements:

-1/numExamples * sum_i (labels[i] * log(predictions[i] + epsilon) + (1-labels[i]) * log(1-predictions[i] + epsilon))

label (NUMERIC) - Label array
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

logPoisson

Log poisson loss: a loss function used for training classifiers.

Implements L = exp(c) - z * c where c is log(predictions) and z is labels.

label (NUMERIC) - Label array. Each value should be 0.0 or 1.0
predictions (NUMERIC) - Predictions array (has to be log(x) of actual predictions)
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

meanPairwiseSquaredError

Mean pairwise squared error. MPWSE loss calculates the difference between pairs of consecutive elements in the predictions and labels arrays.

For example, if predictions = [p0, p1, p2] and labels are [l0, l1, l2] then MPWSE is:

{@code [((p0-p1) - (l0-l1))^2 + ((p0-p2) - (l0-l2))^2 + ((p1-p2) - (l1-l2))^2] / 3}

label (NUMERIC) - Label array
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used. Must be either null, scalar, or have shape [batchSize]

meanSquaredError

Mean squared error loss function. Implements (label[i] - prediction[i])^2 - i.e., squared error on a per-element basis.

When averaged (using LossReduce#MEAN_BY_WEIGHT or LossReduce#MEAN_BY_NONZERO_WEIGHT_COUNT (the default))

this is the mean squared error loss function.

label (NUMERIC) - Label array
predictions (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

sigmoidCrossEntropy

Sigmoid cross entropy: applies the sigmoid activation function on the input logits (input "pre-sigmoid preductions")

and implements the binary cross entropy loss function. This implementation is numerically more stable than using

standard (but separate) sigmoid activation function and log loss (binary cross entropy) loss function. Implements:

-1/numExamples * sum_i (labels[i] * log(sigmoid(logits[i])) + (1-labels[i]) * log(1-sigmoid(logits[i])))

though this is done in a mathematically equivalent but more numerical stable form.

When label smoothing is > 0, the following label smoothing is used:

label (NUMERIC) - Label array
predictionLogits (NUMERIC) - Predictions array
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

softmaxCrossEntropy

Applies the softmax activation function to the input, then implement multi-class cross entropy: {@code -sum_classes label[i] * log(p[c])} where {@code p = softmax(logits)} If LossReduce#NONE is used, returned shape is [numExamples] out for [numExamples, numClasses] predicitons/labels;

otherwise, the output is a scalar.

When label smoothing is > 0, the following label smoothing is used:

oneHotLabels (NUMERIC) - Label array. Should be one-hot per example and same shape as predictions (for example, [mb, nOut])
logitPredictions (NUMERIC) - Predictions array (pre-softmax)
weights (NUMERIC) - Weights array. May be null. If null, a weight of 1.0 is used

sparseSoftmaxCrossEntropy

As per softmaxCrossEntropy(String, SDVariable, SDVariable, LossReduce) but the labels variable

is represented as an integer array instead of the equivalent one-hot array. i.e., if logits are rank N, then labels have rank N-1

logits (NUMERIC) - Logits array ("pre-softmax activations")
labels (INT) - Labels array. Must be an integer type.

weightedCrossEntropyWithLogits

Weighted cross entropy loss with logits

targets (NUMERIC) - targets array
inputs (NUMERIC) - input array
weights (NUMERIC) - eights array. May be null. If null, a weight of 1.0 is used

Random

bernoulli

Generate a new random INDArray, where values are randomly sampled according to a Bernoulli distribution,

with the specified probability. Array values will have value 1 with probability P and value 0 with probability

1-P.

p - Probability of value 1
datatype - Data type of the output variable
shape - Shape of the new random INDArray, as a 1D array (Size: AtLeast(min=0))

binomial

Generate a new random INDArray, where values are randomly sampled according to a Binomial distribution,

with the specified number of trials and probability.

nTrials - Number of trials parameter for the binomial distribution
p - Probability of success for each trial
datatype - Data type of the output variable

exponential

Generate a new random INDArray, where values are randomly sampled according to a exponential distribution:

P(x) = lambda exp(-lambda x)

lambda - lambda parameter
datatype - Data type of the output variable
shape - Shape of the new random INDArray, as a 1D array (Size: AtLeast(min=0))

logNormal

Generate a new random INDArray, where values are randomly sampled according to a Log Normal distribution,

i.e., log(x) ~ N(mean, stdev)

mean - Mean value for the random array
stddev - Standard deviation for the random array
datatype - Data type of the output variable

normal

Generate a new random INDArray, where values are randomly sampled according to a Gaussian (normal) distribution,

N(mean, stdev)

mean - Mean value for the random array
stddev - Standard deviation for the random array
datatype - Data type of the output variable

normalTruncated

Generate a new random INDArray, where values are randomly sampled according to a Gaussian (normal) distribution,

N(mean, stdev). However, any values more than 1 standard deviation from the mean are dropped and re-sampled

mean - Mean value for the random array
stddev - Standard deviation for the random array
datatype - Data type of the output variable

uniform

Generate a new random INDArray, where values are randomly sampled according to a uniform distribution,

U(min,max)

min - Minimum value
max - Maximum value.
datatype - Data type of the output variable

RNN

Operation classes

gru

The GRU operation. Gated Recurrent Unit - Cho et al. 2014.

Variables

What types of variables are used in SameDiff, their properties and how to switch these types.

What are variables

All values defining or passing through each SameDiff instance - be it weights, bias, inputs, activations or general parameters - all are handled by objects of class SDVariable.

Observe that by variables we normally mean not just single values - as it is done in various online examples describing autodifferentiation - but rather whole multidimensional arrays of them.

Variable types

All variables in SameDiff belong to one of four variable types, constituting an enumeration VariableType. Here they are:

VARIABLE: are trainable parameters of your network, e.g. weights and bias of a layer. Naturally, we want them
to be both stored for further usage - we say, that they are persistent - as well as being updated during training.
CONSTANT: are those parameters which, like variables, are persistent for the network, but are not being

To infer the type of a particular variable, you may use the method getVariableType, like so:

The current value of a variable in a form of INDArray may be obtained using getArr or getArr(true) - the latter one if you wish the program to throw an exception if the variable's value is not initialized.

Data types

The data within each variable also has its data type, contained in DataType enum. Currently in DataType there are three floating point types: FLOAT, DOUBLE and HALF; four integer types: LONG, INT, SHORT and UBYTE; one boolean type BOOL - all of them will be referred as numeric types. In addition, there is a string

To infer the data type of your variable, use

You may need to trace your variable's data type since at times it does matter, which types you use in an operation. For example, a convolution product, like this one

will require its SDVariable arguments input and weights to be of one of the floating point data types, and will throw an exception otherwise. Also, as we shall discuss just below, all the SDVariables of type VARIABLE are supposed to be of floating point type.

Common features of variables

Before we go to the differences between variables, let us first look at the properties they all share

All variables are ultimately derived from an instance of SameDiff, serving as parts of its
. In fact, each variable has a SameDiff as one of its fields.
Results (outputs) of all operations are of ARRAY type.

Differences between variable types

Let us now have a closer look at each type of variables, and what distinguish them from each other.

Variables

Variables are the trainable parameters of your network. This predetermines their nature in SameDiff. As we briefly mentioned above, variables' values need to be both preserved for application, and updated during training. Training means, that we iteratively update the values by small fractions of their gradients, and this only makes sense if variables are of floating point types (see data types above).

Variables may be added to your SameDiff using different versions of var function from your SameDiff instance. For example, the code

adds a variable constituting of a 784x10 array of float numbers - weights for a single layer MNIST perceptron in this case - to a pre-existing SameDiff instance samediff.

However, this way the values within a variable will be set as zeros. You may also create a variable with values from a preset INDArray. Say

will create a variable filled with normally distributed randomly generated numbers with variance 1/28. You may put any other array creation methods instead of nrand, or any preset array, of course. Also, you may use some popular initialization scheme, like so:

Now, the weights will be randomly initialized using the Xavier scheme. There are other ways to create and

fill variables: you may look them up in the 'known subclasses' section .

Constants

Constants hold values that are stored, but - unlike variables - remain unchanged during training. These, for instance, may be some hyperparamters you wish to have in your network and be able to access from the outside. Or they may be pretrained weights of a neural network that you wish to keep unchanged (see more on that in below). Constants may be of any data type

so e.g. int and boolean are allowed alongside with float and double.

In general, constants are added to SameDiff by means of constant methods. A constant may be created form an INDArray, like that:

A constant consisting of a single scalar value may be created using one of the scalar methods:

Placeholders

The most common placeholders you'll normally have in a SameDiff are inputs and, when applicable, labels. You may create placeholders of any data type, depending on the operations you use them in. To add a placeholder to a SameDiff, you may call one of placeHolder methods, e.g. like that:

as in MNIST example. Here we specify name, data type and then shape of your placeholder - here, we have 28x28 grayscale pictures rendered as 1d vectors (therefore 784) coming in batches of length we don't know beforehand (therefore -1).

Arrays

Variables of ARRAY type appear as outputs of within SameDiff. Accordingly, the data type of an array-type variable depends on the kind of operation it is produced by and variable type(s) ot its argument(s). Arrays are not persistent - they are one-time values that will be recalculated from scratch at the next step. However, unlike placeholders, gradients are computed for them, as those are needed to update the values of VARIABLE's.

There are as many ways array-type variables are created as there are operations, so you're better up focusing on our , our and .

Recap table

Let us summarize the main properties of variable types in one table:

We haven't discussed what 'Workspaces' mean - if you do not know, do not worry, this is an internal technical term that basically describes how memory is managed internally.

Changing variable types

You may change variable types as well. For now, there are three of such options:

Variable to constant

At times - for instance if you perform transfer learning - you may wish to turn a variable into a constant. This is done like so:

where someVariable is an instance of SDVariable of VARIABLE type. The variable someVariable will not be trained any more.

Constant to variable

Conversely, constants - if they are of floating point data type - may be converted to variables. So, for instance, if you wish your frozen weights to become trainable again

Placeholder to constant

Placeholders may be converted to constants as well - for instance, if you need to freeze one of the inputs. There are no restrictions on the data type, yet, since placeholder values are not persistent, their value should be set before you turn them into constants. This can be done as follows

For now it is not possible to turn a constant back into a placeholder, we may consider adding this functionality if there is a need for that. For now, if you wish to effectively freeze your placeholder but be able to use it again, consider supplying it with constant values rather than turning it into a constant.

Variables' names and values

Getting variables from `SameDiff`

Recall that every variable in an instance of SameDiff has its unique String name. Your SameDiff actually tracks your variables by their names, and allows you to retrieve them by using getVariable(String name) method.

Consider the following line:

Here, in the function sub we actually have implicitly introduced a variable (of type ARRAY) that holds the result of the subtraction. By adding a name into the operations's argument, we've secured ourselves the possibility to retrieve the variable from elsewhere: say, if later you need to infer the difference between the labels and the prediction as a vector, you may just write:

This becomes especially handy if your whole SameDiff instance is initialized elsewhere, and you still need to get hold of some of its variables - say, multiple outputs.

You can get and set the name of an SDVariable the methods getVarName and setVarName respectively. When renaming, note that variable's name is to remain unique within its SameDiff.

Getting variable's value

You may retrieve any variable's current value as an INDArray using the method eval(). Note that for non-persistent variables, the value should first be set. For variables with gradients, the gradient's value may also be inferred using the method getGradient.

NN

CReLU

Concatenates a ReLU which selects only the positive part of the activation with a ReLU which selects only the negative part of the activation. Note that as a result this non-linearity doubles the depth of the activations.

x (NUMERIC) - Input variable

batchNorm

Neural network batch normalization operation.

For details, see

input (NUMERIC) - Input variable.
mean (NUMERIC) - Mean value. For 1d axis, this should match input.size(axis)
variance (NUMERIC) - Variance value. For 1d axis, this should match input.size(axis)

biasAdd

Bias addition operation: a special case of addition, typically used with CNN 4D activations and a 1D bias vector

input (NUMERIC) - 4d input variable
bias (NUMERIC) - 1d bias
nchw - The format - nchw=true means [minibatch, channels, height, width] format; nchw=false - [minibatch, height, width, channels].

dotProductAttention

This operation performs dot product attention on the given timeseries input with the given queries

out = sum(similarity(k_i, q) * v_i)

similarity(k, q) = softmax(k q) where x q is the dot product of x and q

Optionally with normalization step:

similarity(k, q) = softmax(k * q / sqrt(size(q))

See also "Attention is all you need" (, p. 4, eq. 1)

Note: This supports multiple queries at once, if only one query is available the queries vector still has to

be 3D but can have queryCount = 1

Note: keys and values usually is the same array. If you want to use it as the same array, simply pass it for

both.

Note: Queries, keys and values must either be all rank 3 or all rank 4 arrays. Mixing them doesn't work. The

output rank will depend on the input rank.

queries (NUMERIC) - input 3D array "queries" of shape [batchSize, featureKeys, queryCount]
or 4D array of shape [batchSize, numHeads, featureKeys, queryCount]
keys (NUMERIC) - input 3D array "keys" of shape [batchSize, featureKeys, timesteps]
or 4D array of shape [batchSize, numHeads, featureKeys, timesteps]

dropout

Dropout operation

input (NUMERIC) - Input array
inputRetainProbability - Probability of retaining an input (set to 0 with probability 1-p)

elu

Element-wise exponential linear unit (ELU) function:

out = x if x > 0

out = a * (exp(x) - 1) if x <= 0

with constant a = 1.0

See:

x (NUMERIC) - Input variable

gelu

GELU activation function - Gaussian Error Linear Units

For more details, see Gaussian Error Linear Units (GELUs) -

This method uses the sigmoid approximation

x (NUMERIC) - Input variable

hardSigmoid

Element-wise hard sigmoid function:

out[i] = 0 if in[i] <= -2.5

out[1] = 0.2*in[i]+0.5 if -2.5 < in[i] < 2.5

out[i] = 1 if in[i] >= 2.5

x (NUMERIC) - Input variable

hardTanh

Element-wise hard tanh function:

out[i] = -1 if in[i] <= -1

out[1] = in[i] if -1 < in[i] < 1

out[i] = 1 if in[i] >= 1

x (NUMERIC) - Input variable

hardTanhDerivative

Derivative (dOut/dIn) of the element-wise hard Tanh function - hardTanh(INDArray)

x (NUMERIC) - Input variable

layerNorm

Apply Layer Normalization

y = gain * standardize(x) + bias

input (NUMERIC) - Input variable
gain (NUMERIC) - Gain
bias (NUMERIC) - Bias

leakyRelu

Element-wise leaky ReLU function:

out = x if x >= 0.0

out = alpha * x if x < cutoff

Alpha value is most commonly set to 0.01

x (NUMERIC) - Input variable
alpha - Cutoff - commonly 0.01

leakyReluDerivative

Leaky ReLU derivative: dOut/dIn given input.

x (NUMERIC) - Input variable
alpha - Cutoff - commonly 0.01

linear

Linear layer operation: out = mmul(in,w) + bias

Note that bias array is optional

input (NUMERIC) - Input data
weights (NUMERIC) - Weights variable, shape [nIn, nOut]
bias (NUMERIC) - Optional bias variable (may be null)

logSigmoid

Element-wise sigmoid function: out[i] = log(sigmoid(in[i]))

x (NUMERIC) - Input variable

logSoftmax

Log softmax activation

x (NUMERIC) -

logSoftmax

Log softmax activation

x (NUMERIC) - Input
dimension - Dimension along which to apply log softmax

multiHeadDotProductAttention

This performs multi-headed dot product attention on the given timeseries input

out = concat(head_1, head_2, ..., head_n) * Wo

head_i = dot_product_attention(Wq_i_q, Wk_i_k, Wv_i*v)

Optionally with normalization when calculating the attention for each head.

See also "Attention is all you need" (, pp. 4,5, "3.2.2 Multi-Head Attention")

This makes use of dot_product_attention OP support for rank 4 inputs.

see dotProductAttention(INDArray, INDArray, INDArray, INDArray, boolean, boolean)

queries (NUMERIC) - input 3D array "queries" of shape [batchSize, featureKeys, queryCount]
keys (NUMERIC) - input 3D array "keys" of shape [batchSize, featureKeys, timesteps]
values (NUMERIC) - input 3D array "values" of shape [batchSize, featureValues, timesteps]

pad

Padding operation

input (NUMERIC) - Input tensor
padding (NUMERIC) - Padding value
PadMode - Padding format - default = CONSTANT

preciseGelu

GELU activation function - Gaussian Error Linear Units

For more details, see Gaussian Error Linear Units (GELUs) -

This method uses the precise method

x (NUMERIC) - Input variable

prelu

PReLU (Parameterized Rectified Linear Unit) operation. Like LeakyReLU with a learnable alpha:

out[i] = in[i] if in[i] >= 0

out[i] = in[i] * alpha[i] otherwise

sharedAxes allows you to share learnable parameters along axes.

For example, if the input has shape [batchSize, channels, height, width]

and you want each channel to have its own cutoff, use sharedAxes = [2, 3] and an

alpha with shape [channels].

input (NUMERIC) - Input data
alpha (NUMERIC) - The cutoff variable. Note that the batch dimension (the 0th, whether it is batch or not) should not be part of alpha.
sharedAxes - Which axes to share cutoff parameters along. (Size: AtLeast(min=1))

relu

Element-wise rectified linear function with specified cutoff:

out[i] = in[i] if in[i] >= cutoff

out[i] = 0 otherwise

x (NUMERIC) - Input
cutoff - Cutoff value for ReLU operation - x > cutoff ? x : 0. Usually 0

relu6

Element-wise "rectified linear 6" function with specified cutoff:

out[i] = min(max(in, cutoff), 6)

x (NUMERIC) - Input
cutoff - Cutoff value for ReLU operation. Usually 0

reluLayer

ReLU (Rectified Linear Unit) layer operation: out = relu(mmul(in,w) + bias)

Note that bias array is optional

input (NUMERIC) - Input data
weights (NUMERIC) - Weights variable
bias (NUMERIC) - Optional bias variable (may be null)

selu

Element-wise SeLU function - Scaled exponential Lineal Unit: see

out[i] = scale alpha (exp(in[i])-1) if in[i]>0, or 0 if in[i] <= 0

Uses default scale and alpha values.

x (NUMERIC) - Input variable

sigmoid

Element-wise sigmoid function: out[i] = 1.0/(1+exp(-in[i]))

x (NUMERIC) - Input variable

sigmoidDerivative

Element-wise sigmoid function derivative: dL/dIn given input and dL/dOut

x (NUMERIC) - Input Variable
wrt (NUMERIC) - Gradient at the output - dL/dOut. Must have same shape as the input

softmax

Softmax activation, along the specified dimension

x (NUMERIC) - Input
dimension - Dimension along which to apply softmax - default = -1

softmaxDerivative

Softmax derivative function

x (NUMERIC) - Softmax input
wrt (NUMERIC) - Gradient at output, dL/dx
dimension - Softmax dimension

softplus

Element-wise softplus function: out = log(exp(x) + 1)

x (NUMERIC) - Input variable

softsign

Element-wise softsign function: out = x / (abs(x) + 1)

x (NUMERIC) - Input variable

softsignDerivative

Element-wise derivative (dOut/dIn) of the softsign function softsign(INDArray)

x (NUMERIC) - Input variable

swish

Element-wise "swish" function: out = x _sigmoid(b_x) with b=1.0

See:

x (NUMERIC) - Input variable

tanh

Elementwise tanh (hyperbolic tangent) operation: out = tanh(x)

x (NUMERIC) - Input variable

CNN

Operation classes

avgPooling2d

2D Convolution layer operation - average pooling 2d

input (NUMERIC) - the input to average pooling 2d operation - 4d CNN (image) activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
Pooling2DConfig - see

avgPooling3d

3D convolution layer operation - average pooling 3d

input (NUMERIC) - the input to average pooling 3d operation - 5d activations in NCDHW format (shape [minibatch, channels, depth, height, width]) or NDHWC format (shape [minibatch, depth, height, width, channels])
Pooling3DConfig - see

batchToSpace

Convolution 2d layer batch to space operation on 4d input.

Reduces input batch dimension by rearranging data into a larger spatial dimensions

x (NUMERIC) - Input variable. 4d input
blocks - Block size, in the height/width dimension (Size: Exactly(count=2))
croppingTop - (Size: Exactly(count=2))

col2Im

col2im operation for use in 2D convolution operations. Outputs a 4d array with shape

[minibatch, inputChannels, height, width]

in (NUMERIC) - Input - rank 6 input with shape [minibatch, inputChannels, kernelHeight, kernelWidth, outputHeight, outputWidth]
Conv2DConfig - see

conv1d

Conv1d operation.

input (NUMERIC) - the inputs to conv1d
weights (NUMERIC) - weights for conv1d op - rank 3 array with shape [kernelSize, inputChannels, outputChannels]
bias (NUMERIC) - bias for conv1d op - rank 1 array with shape [outputChannels]. May be null.

conv2d

2D Convolution operation with optional bias

layerInput (NUMERIC) - the input to max pooling 2d operation - 4d CNN (image) activations in NCHW format
weights (NUMERIC) - Weights for the convolution operation. 4 dimensions with format [kernelHeight, kernelWidth, inputChannels, outputChannels]
bias (NUMERIC) - Optional 1D bias array with shape [outputChannels]. May be null.

conv3d

Convolution 3D operation with optional bias

input (NUMERIC) - the input to average pooling 3d operation - 5d activations in NCDHW format (shape [minibatch, channels, depth, height, width]) or NDHWC format (shape [minibatch, depth, height, width, channels])
weights (NUMERIC) - Weights for conv3d. Rank 5 with shape [kernelDepth, kernelHeight, kernelWidth, inputChannels, outputChannels].
bias (NUMERIC) - Optional 1D bias array with shape [outputChannels]. May be null.

deconv2d

2D deconvolution operation with optional bias

layerInput (NUMERIC) - the input to deconvolution 2d operation - 4d CNN (image) activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
weights (NUMERIC) - Weights for the 2d deconvolution operation. 4 dimensions with format [inputChannels, outputChannels, kernelHeight, kernelWidth]
bias (NUMERIC) - Optional 1D bias array with shape [outputChannels]. May be null.

deconv3d

3D CNN deconvolution operation with or without optional bias

input (NUMERIC) - Input array - shape [bS, iD, iH, iW, iC] (NDHWC) or [bS, iC, iD, iH, iW] (NCDHW)
weights (NUMERIC) - Weights array - shape [kD, kH, kW, oC, iC]
bias (NUMERIC) - Bias array - optional, may be null. If non-null, must have shape [outputChannels]

depthToSpace

Convolution 2d layer batch to space operation on 4d input. Reduces input channels dimension by rearranging data into a larger spatial dimensions Example: if input has shape [mb, 8, 2, 2] and block size is 2, then output size is [mb, 8/(2_2), 2_2, 2*2]

= [mb, 2, 4, 4]

x (NUMERIC) - the input to depth to space pooling 2d operation - 4d activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
blockSize - Block size, in the height/width dimension
dataFormat - Data format: "NCHW" or "NHWC"

depthWiseConv2d

Depth-wise 2D convolution operation with optional bias

layerInput (NUMERIC) - the input to max pooling 2d operation - 4d CNN (image) activations in NCHW format
depthWeights (NUMERIC) - Depth-wise conv2d weights. 4 dimensions with format [kernelHeight, kernelWidth, inputChannels, depthMultiplier]
bias (NUMERIC) - Optional 1D bias array with shape [outputChannels]. May be null.

dilation2D

TODO doc string

df (NUMERIC) -
weights (NUMERIC) - df
strides - weights (Size: Exactly(count=2))

extractImagePatches

Extract image patches

input (NUMERIC) - Input array. Must be rank 4, with shape [minibatch, height, width, channels]
kH - Kernel height
kW - Kernel width

im2Col

im2col operation for use in 2D convolution operations. Outputs a 6d array with shape

[minibatch, inputChannels, kernelHeight, kernelWidth, outputHeight, outputWidth]

in (NUMERIC) - Input - rank 4 input with shape [minibatch, inputChannels, height, width]
Conv2DConfig - see

localResponseNormalization

2D convolution layer operation - local response normalization

input (NUMERIC) - the inputs to lrn
LocalResponseNormalizationConfig - see

maxPoolWithArgmax

2D Convolution layer operation - Max pooling on the input and outputs both max values and indices

input (NUMERIC) - the input to max pooling 2d operation - 4d CNN (image) activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
Pooling2DConfig - see

maxPooling2d

2D Convolution layer operation - max pooling 2d

input (NUMERIC) - the input to max pooling 2d operation - 4d CNN (image) activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
Pooling2DConfig - see

maxPooling3d

3D convolution layer operation - max pooling 3d operation.

input (NUMERIC) - the input to average pooling 3d operation - 5d activations in NCDHW format (shape [minibatch, channels, depth, height, width]) or NDHWC format (shape [minibatch, depth, height, width, channels])
Pooling3DConfig - see

separableConv2d

Separable 2D convolution operation with optional bias

layerInput (NUMERIC) - the input to max pooling 2d operation - 4d CNN (image) activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
depthWeights (NUMERIC) - Separable conv2d depth weights. 4 dimensions with format [kernelHeight, kernelWidth, inputChannels, depthMultiplier]
pointWeights (NUMERIC) - Point weights, rank 4 with format [1, 1, inputChannels*depthMultiplier, outputChannels]. May be null

spaceToBatch

Convolution 2d layer space to batch operation on 4d input.

Increases input batch dimension by rearranging data from spatial dimensions into batch dimension

x (NUMERIC) - Input variable. 4d input
blocks - Block size, in the height/width dimension (Size: Exactly(count=2))
paddingTop - Optional 2d int[] array for padding the result: values [[pad top, pad bottom], [pad left, pad right]] (Size: Exactly(count=2))

spaceToDepth

Convolution 2d layer space to depth operation on 4d input. Increases input channels (reduced spatial dimensions) by rearranging data into a larger channels dimension Example: if input has shape [mb, 2, 4, 4] and block size is 2, then output size is [mb, 8/(2_2), 2_2, 2*2]

= [mb, 2, 4, 4]

x (NUMERIC) - the input to depth to space pooling 2d operation - 4d activations in NCHW format (shape [minibatch, channels, height, width]) or NHWC format (shape [minibatch, height, width, channels])
blockSize - Block size, in the height/width dimension
dataFormat - Data format: "NCHW" or "NHWC"

upsampling2d

Upsampling layer for 2D inputs.

scale is used for both height and width dimensions.

input (NUMERIC) - Input in NCHW format
scale - The scale for both height and width dimensions.

upsampling2d

2D Convolution layer operation - Upsampling 2d

input (NUMERIC) - Input in NCHW format
scaleH - Scale to upsample in height dimension
scaleW - Scale to upsample in width dimension

upsampling3d

3D Convolution layer operation - Upsampling 3d

input (NUMERIC) - Input in NCHW format
ncdhw - If true: input is in NCDHW (minibatch, channels, depth, height, width) format. False: NDHWC format
scaleD - Scale to upsample in depth dimension

Configuration Classes

Conv1DConfig

k (LONG) - Kernel - default = -1
s (LONG) - stride - default = 1
p (LONG) - padding - default = 0

Used in these ops:

Conv2DConfig

kH (LONG) - Kernel height - default = -1
kW (LONG) - Kernel width - default = -1
sH (LONG) - Stride along height dimension - default = 1

Used in these ops:

Conv3DConfig

kD (LONG) - Kernel depth - default = -1
kW (LONG) - Kernel width - default = -1
kH (LONG) - Kernel height - default = -1

Used in these ops:

DeConv2DConfig

kH (LONG) - Kernel height - default = -1
kW (LONG) - Kernel width - default = -1
sH (LONG) - Stride along height dimension - default = 1

Used in these ops:

DeConv3DConfig

kD (LONG) - Kernel depth - default = -1
kW (LONG) - Kernel width - default = -1
kH (LONG) - Kernel height - default = -1

Used in these ops:

Pooling2DConfig

kH (LONG) - Kernel height - default = -1
kW (LONG) - Kernel width - default = -1
sH (LONG) - Stride along height dimension - default = 1

Used in these ops:

Pooling3DConfig

kD (LONG) - Kernel depth - default = -1
kW (LONG) - Kernel width - default = -1
kH (LONG) - Kernel height - default = -1

Used in these ops:

LocalResponseNormalizationConfig

alpha (NUMERIC) - alpha - default = 1
beta (NUMERIC) - beta - default = 0.5
bias (NUMERIC) - bias - default = 1

Used in these ops:

Base Operations

These ops are generally available directly on SameDiff instances. Due to an oversight before the release, this ops aren't also available on Nd4j. To use the INDArray variants of these operations, you will have to instantiate a NDBase instance.

all

Boolean and array reduction operation, optionally along specified dimensions

x (NDARRAY) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

any

Boolean or array reduction operation, optionally along specified dimensions

x (NDARRAY) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

argmax

Argmax array reduction operation, optionally along specified dimensions.

Output values are the index of the maximum value of each slice along the specified dimension.

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

in (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

argmin

Argmin array reduction operation, optionally along specified dimensions.

Output values are the index of the minimum value of each slice along the specified dimension.

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

in (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

batchMmul

Matrix multiply a batch of matrices. matricesA and matricesB have to be arrays of same

length and each pair taken from these sets has to have dimensions (M, N) and (N, K),

respectively. If transposeA is true, matrices from matricesA will have shape (N, M) instead.

Likewise, if transposeB is true, matrices from matricesB will have shape (K, N).

The result of this operation will be a batch of multiplied matrices. The

result has the same length as both input batches and each output matrix is of shape (M, K).

inputsA (NUMERIC) - First array of input matrices, all of shape (M, N) or (N, M)
inputsB (NUMERIC) - Second array of input matrices, all of shape (N, K) or (K, N)
transposeA - Whether to transpose A arrays or not - default = false

castTo

Cast the array to a new datatype - for example, Integer -> Float

arg (NDARRAY) - Input variable to cast
datatype - Datatype to cast to

concat

Concatenate a set of inputs along the specified dimension.

Note that inputs must have identical rank and identical dimensions, other than the dimension to stack on.

For example, if 2 inputs have shape [a, x, c] and [a, y, c] and dimension = 1, then the output has shape [a, x+y, c]

inputs (NUMERIC) - Input variables
dimension - Dimension to concatenate on

cumprod

Cumulative product operation.

For input: [ a, b, c], output is:

exclusive=false, reverse=false: [a, a_b, a_b*c]

exclusive=true, reverse=false, [0, a, a*b]

exclusive=false, reverse=true: [a_b_c, b*c, c]

exclusive=true, reverse=true: [b*c, c, 0]

in (NUMERIC) - Input variable
exclusive - If true: exclude the first value - default = false
reverse - If true: reverse the direction of the accumulation - default = false

cumsum

Cumulative sum operation.

For input: [ a, b, c], output is:

exclusive=false, reverse=false: [a, a+b, a+b+c]

exclusive=true, reverse=false, [0, a, a+b]

exclusive=false, reverse=true: [a+b+c, b+c, c]

exclusive=true, reverse=true: [b+c, c, 0]

in (NUMERIC) - Input variable
exclusive - If true: exclude the first value - default = false
reverse - If true: reverse the direction of the accumulation - default = false

dot

Pairwise dot product reduction along dimension

output = sum(i=0 ... size(dim)-1) x[i] * y[i]

x (NUMERIC) - first input
y (NUMERIC) - second input
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

dynamicPartition

Dynamically partition the input variable values into the specified number of paritions, using the indices.

Example:

x (NUMERIC) - Input variable
partitions (INT) - 1D input with values 0 to numPartitions-1
numPartitions - Number of partitions, >= 1

dynamicStitch

Dynamically merge the specified input arrays into a single array, using the specified indices

indices (INT) - Indices to use when merging. Must be >= 1, same length as input variables
x (NUMERIC) - Input variables.

eq

Equals operation: elementwise x == y

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input array
y - Double value argument to use in operation

eq

Equal to operation: elementwise x == y

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input 1
y (NUMERIC) - Input 2

expandDims

Reshape the input by adding a 1 at the specified location.

For example, if input has shape [a, b], then output shape is:

axis = 0: [1, a, b]

axis = 1: [a, 1, b]

axis = 2: [a, b, 1]

x (NDARRAY) - Input variable
axis - Axis to expand

fill

Generate an output variable with the specified (dynamic) shape with all elements set to the specified value

shape (INT) - Shape: must be a 1D array/variable
dataType - Datatype of the output array
value - Value to set all elements to

gather

Gather slices from the input variable where the indices are specified as fixed int[] values.

Output shape is same as input shape, except for axis dimension, which has size equal to indices.length.

df (NUMERIC) - Input variable
indices - Indices to get (Size: AtLeast(min=1))
axis - Axis that the indices refer to

gather

Gather slices from the input variable where the indices are specified as dynamic array values.

Output shape is same as input shape, except for axis dimension, which has size equal to indices.length.

df (NUMERIC) - Input variable
indices (INT) - Indices to get slices for. Rank 0 or 1 input
axis - Axis that the indices refer to

gatherNd

Gather slices from df with shape specified by indices.

df (NUMERIC) -
indices (NUMERIC) -

gt

Greater than operation: elementwise x > y

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input array
y - Double value argument to use in operation

gt

Greater than operation: elementwise x > y

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input 1
y (NUMERIC) - Input 2

gte

Greater than or equals operation: elementwise x >= y

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input array
y - Double value argument to use in operation

gte

Greater than or equal to operation: elementwise x >= y

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input 1
y (NUMERIC) - Input 2

identity

Elementwise identity operation: out = x

input (NUMERIC) - Input variable

invertPermutation

Compute the inverse permutation indices for a permutation operation

Example: if input is [2, 0, 1] then output is [1, 2, 0]

The idea is that x.permute(input).permute(invertPermutation(input)) == x

input (INT) - 1D indices for permutation

isNumericTensor

Is the director a numeric tensor? In the current version of ND4J/SameDiff, this always returns true/1

x (NUMERIC) - Input variable

linspace

Create a new 1d array with values evenly spaced between values 'start' and 'stop'

For example, linspace(start=3.0, stop=4.0, number=3) will generate [3.0, 3.5, 4.0]

dataType - Data type of the output array
start - Start value
stop - Stop value

linspace

Create a new 1d array with values evenly spaced between values 'start' and 'stop'

For example, linspace(start=3.0, stop=4.0, number=3) will generate [3.0, 3.5, 4.0]

start (NUMERIC) - Start value
stop (NUMERIC) - Stop value
number (LONG) - Number of values to generate

lt

Less than operation: elementwise x < y

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input array
y - Double value argument to use in operation

lt

Less than operation: elementwise x < y

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input 1
y (NUMERIC) - Input 2

lte

Less than or equals operation: elementwise x <= y

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input array
y - Double value argument to use in operation

lte

Less than or equal to operation: elementwise x <= y

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input 1
y (NUMERIC) - Input 2

matchCondition

Returns a boolean mask of equal shape to the input, where the condition is satisfied - value 1 where satisfied, 0 otherwise

in (NUMERIC) - Input
condition - Condition

matchConditionCount

Returns a count of the number of elements that satisfy the condition

in (NUMERIC) - Input
condition - Condition

matchConditionCount

Returns a count of the number of elements that satisfy the condition (for each slice along the specified dimensions)

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

in (NUMERIC) - Input variable
condition - Condition
keepDim - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false

max

Max array reduction operation, optionally along specified dimensions

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

max

Element-wise maximum operation: out[i] = max(first[i], second[i])

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

first (NUMERIC) - First input array
second (NUMERIC) - Second input array

mean

Mean (average) array reduction operation, optionally along specified dimensions

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

merge

The merge operation is a control operation that forwards the either of the inputs to the output, when

the first of them becomes available. If both are available, the output is undefined (either input could

be forwarded to the output)

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

min

Minimum array reduction operation, optionally along specified dimensions. out = min(in)

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

min

Element-wise minimum operation: out[i] = min(first[i], second[i])

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

first (NUMERIC) - First input array
second (NUMERIC) - Second input array

mmul

Matrix multiplication: out = mmul(x,y)

Supports specifying transpose argument to perform operation such as mmul(a^T, b), etc.

x (NUMERIC) - First input variable
y (NUMERIC) - Second input variable
transposeX - Transpose x (first argument) - default = false

neq

Not equals operation: elementwise x != y

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input array
y - Double value argument to use in operation

neq

Not equal to operation: elementwise x != y

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

Return boolean array with values true where satisfied, or false otherwise.

x (NUMERIC) - Input 1
y (NUMERIC) - Input 2

norm1

Norm1 (L1 norm) reduction operation: The output contains the L1 norm for each tensor/subset along the specified dimensions:

out = sum_i abs(x[i])

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - dimensions to reduce over (Size: AtLeast(min=0))

norm2

Norm2 (L2 norm) reduction operation: The output contains the L2 norm for each tensor/subset along the specified dimensions:

out = sqrt(sum_i x[i]^2)

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - dimensions dimensions to reduce over (Size: AtLeast(min=0))

normmax

Max norm (infinity norm) reduction operation: The output contains the max norm for each tensor/subset along the

specified dimensions:

out = max(abs(x[i]))

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - dimensions to reduce over (Size: AtLeast(min=0))

oneHot

Convert the array to a one-hot array with walues and for each entry

If input has shape [ a, ..., n] then output has shape [ a, ..., n, depth],

with {out[i, ..., j, in[i,...,j]] with other values being set to

indices (NUMERIC) - Indices - value 0 to depth-1
depth - Number of classes
axis -

oneHot

Convert the array to a one-hot array with walues 0 and 1 for each entry

If input has shape [ a, ..., n] then output has shape [ a, ..., n, depth],

with out[i, ..., j, in[i,...,j]] = 1 with other values being set to 0

see oneHot(SDVariable, int, int, double, double)

indices (NUMERIC) - Indices - value 0 to depth-1
depth - Number of classes

onesLike

Return a variable of all 1s, with the same shape as the input variable. Note that this is dynamic:

if the input shape changes in later execution, the returned variable's shape will also be updated

input (NUMERIC) - Input INDArray

onesLike

As per onesLike(String, SDVariable) but the output datatype may be specified

input (NUMERIC) -
dataType -

permute

Array permutation operation: permute the dimensions according to the specified permutation indices.

Example: if input has shape [a,b,c] and dimensions = [2,0,1] the output has shape [c,a,b]

x (NUMERIC) - Input variable
dimensions (INT) - Permute dimensions

permute

Array permutation operation: permute the dimensions according to the specified permutation indices.

Example: if input has shape [a,b,c] and dimensions = [2,0,1] the output has shape [c,a,b]

x (NUMERIC) - Input variable
dimensions - (Size: AtLeast(min=0))

prod

Product array reduction operation, optionally along specified dimensions

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

range

Create a new variable with a 1d array, where the values start at from and increment by step

up to (but not including) limit.

For example, range(1.0, 3.0, 0.5) will return [1.0, 1.5, 2.0, 2.5]

from - Initial/smallest value
to - Largest value (exclusive)
step - Step size

range

Create a new variable with a 1d array, where the values start at from and increment by step

up to (but not including) limit.

For example, range(1.0, 3.0, 0.5) will return [1.0, 1.5, 2.0, 2.5]

from (NUMERIC) - Initial/smallest value
to (NUMERIC) - Largest value (exclusive)
step (NUMERIC) - Step size

rank

Returns the rank (number of dimensions, i.e., length(shape)) of the specified INDArray as a 0D scalar variable

in (NUMERIC) - Input variable

replaceWhere

Element-wise replace where condition:

out[i] = from[i] if condition(update[i]) is satisfied, or

out[i] = update[i] if condition(update[i]) is NOT satisfied

update (NUMERIC) - Source array
from (NUMERIC) - Replacement values array (used conditionally). Must be same shape as 'update' array
condition - Condition to check on update array elements

replaceWhere

Element-wise replace where condition:

out[i] = value if condition(update[i]) is satisfied, or

out[i] = update[i] if condition(update[i]) is NOT satisfied

update (NUMERIC) - Source array
value - Value to set at the output, if the condition is satisfied
condition - Condition to check on update array elements

reshape

Reshape the input variable to the specified (fixed) shape. The output variable will have the same values as the

input, but with the specified shape.

Note that prod(shape) must match length(input) == prod(input.shape)

x (NUMERIC) - Input variable
shape (NUMERIC) - New shape for variable

reshape

Reshape the input variable to the specified (fixed) shape. The output variable will have the same values as the

input, but with the specified shape.

Note that prod(shape) must match length(input) == prod(input.shape)

x (NUMERIC) - Input variable
shape - New shape for variable (Size: AtLeast(min=0))

reverse

Reverse the values of an array for the specified dimensions

If input is:

[ 1, 2, 3]

[ 4, 5, 6]

then

reverse(in, 0):

[3, 2, 1]

[6, 5, 4]

reverse(in, 1):

[4, 5, 6]

[1, 2 3]

x (NUMERIC) - Input variable
dimensions - Input variable (Size: AtLeast(min=0))

reverseSequence

Reverse sequence op: for each slice along dimension seqDimension, the first seqLength values are reversed

x (NUMERIC) - Input variable
seq_lengths (INT) - Length of the sequences
seqDim - Sequence dimension - default = -1

scalarFloorMod

Element-wise scalar floor modulus operation: out = floorMod(in, value).

i.e., returns the remainder after division by 'value'

in (NUMERIC) - Input variable
value - Scalar value to compare

scalarMax

Element-wise scalar maximum operation: out = max(in, value)

in (NUMERIC) - Input variable
value - Scalar value to compare

scalarMin

Element-wise scalar minimum operation: out = min(in, value)

in (NUMERIC) - Input variable
value - Scalar value to compare

scalarSet

Return a variable with equal shape to the input, but all elements set to value 'set'

in (NUMERIC) - Input variable
set - Value to set

scatterAdd

Scatter addition operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

scatterDiv

Scatter division operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

scatterMax

Scatter max operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

scatterMin

Scatter min operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

scatterMul

Scatter multiplication operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

scatterSub

Scatter subtraction operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

scatterUpdate

Scatter update operation.

If indices is rank 0 (a scalar), then out[index, ...] = out[index, ...] + op(updates[...])

If indices is rank 1 (a vector), then for each position i, out[indices[i], ...] = out[indices[i], ...] + op(updates[i, ...])

If indices is rank 2+, then for each position (i,...,k), out[indices[i], ..., indices[k], ...] = out[indices[i], ..., indices[k], ...] + op(updates[i, ..., k, ...])

Note that if multiple indices refer to the same location, the contributions from each is handled correctly.

ref (NUMERIC) - Initial/source variable
indices (NUMERIC) - Indices array
updates (NUMERIC) - Updates to add to the initial/source array

segmentMax

Segment max operation.

If data = [3, 6, 1, 4, 9, 2, 8]

segmentIds = [0, 0, 1, 1, 1, 2, 2]

then output = [6, 9, 8] = [op(3,6), op(1,4,9), op(2,8)]

Note that the segment IDs must be sorted from smallest to largest segment.

See {unsortedSegment (String, SDVariable, SDVariable, int) ops

for the same op without this sorted requirement

data (NDARRAY) - Data to perform segment max on
segmentIds (NUMERIC) - Variable for the segment IDs

segmentMean

Segment mean operation.

If data = [3, 6, 1, 4, 9, 2, 8]

segmentIds = [0, 0, 1, 1, 1, 2, 2]

then output = [6, 9, 8] = [op(3,6), op(1,4,9), op(2,8)]

Note that the segment IDs must be sorted from smallest to largest segment.

See {unsortedSegment (String, SDVariable, SDVariable, int) ops

for the same op without this sorted requirement

data (NDARRAY) - Data to perform segment max on
segmentIds (NUMERIC) - Variable for the segment IDs

segmentMin

Segment min operation.

If data = [3, 6, 1, 4, 9, 2, 8]

segmentIds = [0, 0, 1, 1, 1, 2, 2]

then output = [6, 9, 8] = [op(3,6), op(1,4,9), op(2,8)]

Note that the segment IDs must be sorted from smallest to largest segment.

See {unsortedSegment (String, SDVariable, SDVariable, int) ops

for the same op without this sorted requirement

data (NDARRAY) - Data to perform segment max on
segmentIds (NUMERIC) - Variable for the segment IDs

segmentProd

Segment product operation.

If data = [3, 6, 1, 4, 9, 2, 8]

segmentIds = [0, 0, 1, 1, 1, 2, 2]

then output = [6, 9, 8] = [op(3,6), op(1,4,9), op(2,8)]

Note that the segment IDs must be sorted from smallest to largest segment.

See {unsortedSegment (String, SDVariable, SDVariable, int) ops

for the same op without this sorted requirement

data (NDARRAY) - Data to perform segment max on
segmentIds (NUMERIC) - Variable for the segment IDs

segmentSum

Segment sum operation.

If data = [3, 6, 1, 4, 9, 2, 8]

segmentIds = [0, 0, 1, 1, 1, 2, 2]

then output = [6, 9, 8] = [op(3,6), op(1,4,9), op(2,8)]

Note that the segment IDs must be sorted from smallest to largest segment.

See {unsortedSegment (String, SDVariable, SDVariable, int) ops

for the same op without this sorted requirement

data (NDARRAY) - Data to perform segment max on
segmentIds (NUMERIC) - Variable for the segment IDs

sequenceMask

Generate a sequence mask (with values 0 or 1) based on the specified lengths

Specifically, out[i, ..., k, j] = (j < lengths[i, ..., k] ? 1.0 : 0.0)

lengths (NUMERIC) - Lengths of the sequences
maxLen - Maximum sequence length
dataType -

sequenceMask

Generate a sequence mask (with values 0 or 1) based on the specified lengths

Specifically, out[i, ..., k, j] = (j < lengths[i, ..., k] ? 1.0 : 0.0)

lengths (NUMERIC) - Lengths of the sequences
maxLen (INT) - Maximum sequence length
dataType -

sequenceMask

see sequenceMask(String, SDVariable, SDVariable, DataType)

lengths (NUMERIC) -
dataType -

shape

Returns the shape of the specified INDArray as a 1D INDArray

input (NUMERIC) - Input variable

size

Returns the size (number of elements, i.e., prod(shape)) of the specified INDArray as a 0D scalar variable

in (NUMERIC) - Input variable

sizeAt

Returns a rank 0 (scalar) variable for the size of the specified dimension.

For example, if X has shape [10,20,30] then sizeAt(X,1)=20. Similarly, sizeAt(X,-1)=30

in (NUMERIC) - Input variable
dimension - Dimension to get size of

slice

Get a subset of the specified input, by specifying the first element and the size of the array.

For example, if input is:

[a, b, c]

[d, e, f]

then slice(input, begin=[0,1], size=[2,1] will return:

[b]

[e]

Note that for each dimension i, begin[i] + size[i] <= input.size(i)

input (NUMERIC) - input Variable to get subset of
begin - Beginning index. Must be same length as rank of input array (Size: AtLeast(min=1))
size - Size of the output array. Must be same length as rank of input array (Size: AtLeast(min=1))

slice

Get a subset of the specified input, by specifying the first element and the size of the array.

For example, if input is:

[a, b, c]

[d, e, f]

then slice(input, begin=[0,1], size=[2,1] will return:

[b]

[e]

Note that for each dimension i, begin[i] + size[i] <= input.size(i)

input (NUMERIC) - input Variable to get subset of
begin (INT) - Beginning index. Must be same length as rank of input array
size (INT) - Size of the output array. Must be same length as rank of input array

squaredNorm

Squared L2 norm: see norm2(String, SDVariable, boolean, int...)

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) -
keepDims - - default = false
dimensions - (Size: AtLeast(min=0))

squeeze

Remove a single dimension of size 1.

For example, if input has shape [a,b,1,c] then squeeze(input, 2) returns an array of shape [a,b,c]

x (NUMERIC) - Input variable
axis - Size 1 dimension to remove

stack

Stack a set of N INDArray of rank X into one rank X+1 variable.

If inputs have shape [a,b,c] then output has shape:

axis = 0: [N,a,b,c]

axis = 1: [a,N,b,c]

axis = 2: [a,b,N,c]

axis = 3: [a,b,c,N]

see unstack(String[], SDVariable, int, int)

values (NDARRAY) - Input variables to stack. Must have the same shape for all inputs
axis - Axis to stack on

standardDeviation

Stardard deviation array reduction operation, optionally along specified dimensions

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
biasCorrected - If true: divide by (N-1) (i.e., sample stdev). If false: divide by N (population stdev)
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false

stridedSlice

Get a subset of the specified input, by specifying the first element, last element, and the strides.

For example, if input is:

[a, b, c]

[d, e, f]

[g, h, i]

then stridedSlice(input, begin=[0,1], end=[2,2], strides=[2,1], all masks = 0) will return:

[b, c]

[h, i]

in (NUMERIC) - Variable to get subset of
begin - Beginning index (Size: AtLeast(min=1))
end - End index (Size: AtLeast(min=1))

sum

Sum array reduction operation, optionally along specified dimensions.

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
keepDims - If true: keep the dimensions that are reduced on (as length 1). False: remove the reduction dimensions - default = false
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

switchOp

Switch operation

Predictate - if false, values are output to left (first) branch/output; if true, to right (second) branch/output

x (NDARRAY) - Input variable
predicate (BOOL) - Predictate - if false, values are output to left (first) branch/output; if true, to right (second) branch/output

tensorMmul

//TODO: Ops must be documented.

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensionsX - dimensions for first input array (x) (Size: AtLeast(min=1))

tile

Repeat (tile) the input tensor the specified number of times.

For example, if input is

[1, 2]

[3, 4]

and repeat is [2, 3]

then output is

[1, 2, 1, 2, 1, 2]

[3, 4, 3, 4, 3, 4]

[1, 2, 1, 2, 1, 2]

[3, 4, 3, 4, 3, 4]

x (NDARRAY) - Input variable
repeat (INT) - Number of times to repeat in each axis. Must have length equal to the rank of the input array

tile

see tile(String, SDVariable, int...)

x (NDARRAY) -
repeat - (Size: AtLeast(min=1))

transpose

Matrix transpose operation: If input has shape [a,b] output has shape [b,a]

x (NDARRAY) - Input variable

unsortedSegmentMax

Unsorted segment max operation. As per segmentMax(String, SDVariable, SDVariable) but without

the requirement for the indices to be sorted.

If data = [1, 3, 2, 6, 4, 9, 8]

segmentIds = [1, 0, 2, 0, 1, 1, 2]

then output = [6, 9, 8] = [max(3,6), max(1,4,9), max(2,8)]

data (NUMERIC) - Data (variable) to perform unsorted segment max on
segmentIds (NUMERIC) - Variable for the segment IDs
numSegments - Number of segments

unsortedSegmentMean

Unsorted segment mean operation. As per segmentMean(String, SDVariable, SDVariable) but without

the requirement for the indices to be sorted.

If data = [1, 3, 2, 6, 4, 9, 8]

segmentIds = [1, 0, 2, 0, 1, 1, 2]

then output = [4.5, 4.666, 5] = [mean(3,6), mean(1,4,9), mean(2,8)]

data (NUMERIC) - Data (variable) to perform unsorted segment max on
segmentIds (NUMERIC) - Variable for the segment IDs
numSegments - Number of segments

unsortedSegmentMin

Unsorted segment min operation. As per segmentMin(String, SDVariable, SDVariable) but without

the requirement for the indices to be sorted.

If data = [1, 3, 2, 6, 4, 9, 8]

segmentIds = [1, 0, 2, 0, 1, 1, 2]

then output = [3, 1, 2] = [min(3,6), min(1,4,9), min(2,8)]

data (NUMERIC) - Data (variable) to perform unsorted segment max on
segmentIds (NUMERIC) - Variable for the segment IDs
numSegments - Number of segments

unsortedSegmentProd

Unsorted segment product operation. As per segmentProd(String, SDVariable, SDVariable) but without

the requirement for the indices to be sorted.

If data = [1, 3, 2, 6, 4, 9, 8]

segmentIds = [1, 0, 2, 0, 1, 1, 2]

then output = [4.5, 4.666, 5] = [mean(3,6), mean(1,4,9), mean(2,8)]

data (NUMERIC) - Data (variable) to perform unsorted segment max on
segmentIds (NUMERIC) - Variable for the segment IDs
numSegments - Number of segments

unsortedSegmentSqrtN

Unsorted segment sqrtN operation. Simply returns the sqrt of the count of the number of values in each segment

If data = [1, 3, 2, 6, 4, 9, 8]

segmentIds = [1, 0, 2, 0, 1, 1, 2]

then output = [1.414, 1.732, 1.414] = [sqrt(2), sqrtN(3), sqrtN(2)]

data (NUMERIC) - Data (variable) to perform unsorted segment max on
segmentIds (NUMERIC) - Variable for the segment IDs
numSegments - Number of segments

unsortedSegmentSum

Unsorted segment sum operation. As per segmentSum(String, SDVariable, SDVariable) but without

the requirement for the indices to be sorted.

If data = [1, 3, 2, 6, 4, 9, 8]

segmentIds = [1, 0, 2, 0, 1, 1, 2]

then output = [9, 14, 10] = [sum(3,6), sum(1,4,9), sum(2,8)]

data (NUMERIC) - Data (variable) to perform unsorted segment max on
segmentIds (NUMERIC) - Variable for the segment IDs
numSegments - Number of segments

unstack

Unstack a variable of rank X into N rank X-1 variables by taking slices along the specified axis.

If input has shape [a,b,c] then output has shape:

axis = 0: [b,c]

axis = 1: [a,c]

axis = 2: [a,b]

value (NDARRAY) - Input variable to unstack
axis - Axis to unstack on
num - Number of output variables

variance

Variance array reduction operation, optionally along specified dimensions

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

x (NUMERIC) - Input variable
biasCorrected - If true: divide by (N-1) (i.e., sample variable). If false: divide by N (population variance)
keepDims - If true: keep the dimensions that are reduced on (as size 1). False: remove the reduction dimensions - default = false

zerosLike

Return a variable of all 0s, with the same shape as the input variable. Note that this is dynamic:

if the input shape changes in later execution, the returned variable's shape will also be updated

input (NUMERIC) - Input

Math

ClipByAvgNorm

Clips tensor values to a maximum average L2-norm.

x (NUMERIC) - Input variable
clipValue - Value for clipping
dimensions - Dimensions to reduce over (Size: AtLeast(min=0))

EmbeddingLookup

Looks up ids in a list of embedding tensors.

x (NUMERIC) - Input tensor
indices (INT) - A Tensor containing the ids to be looked up.
PartitionMode - partition_mode == 0 - i.e. 'mod' , 1 - 'div'

MergeMaxIndex

Return array of max elements indices with along tensor dimensions

x (NUMERIC) - Input tensor
dataType - Data type - default = DataType.INT

abs

Elementwise absolute value operation: out = abs(x)

x (NUMERIC) - Input variable

acos

Elementwise acos (arccosine, inverse cosine) operation: out = arccos(x)

x (NUMERIC) - Input variable

acosh

Elementwise acosh (inverse hyperbolic cosine) function: out = acosh(x)

x (NUMERIC) - Input variable

add

Pairwise addition operation, out = x + y

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

add

Scalar add operation, out = in + scalar

x (NUMERIC) - Input variable
value - Scalar value for op

amax

Absolute max array reduction operation, optionally along specified dimensions: out = max(abs(x))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

amean

Absolute mean array reduction operation, optionally along specified dimensions: out = mean(abs(x))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

amin

Absolute min array reduction operation, optionally along specified dimensions: out = min(abs(x))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

and

Boolean AND operation: elementwise (x != 0) && (y != 0)

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

Returns an array with values 1 where condition is satisfied, or value 0 otherwise.

x (BOOL) - Input 1
y (BOOL) - Input 2

asin

Elementwise asin (arcsin, inverse sine) operation: out = arcsin(x)

x (NUMERIC) - Input variable

asinh

Elementwise asinh (inverse hyperbolic sine) function: out = asinh(x)

x (NUMERIC) - Input variable

asum

Absolute sum array reduction operation, optionally along specified dimensions: out = sum(abs(x))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

atan

Elementwise atan (arctangent, inverse tangent) operation: out = arctangent(x)

x (NUMERIC) - Input variable

atan2

Elementwise atan (arctangent, inverse tangent) operation: out = atan2(x,y).

Similar to atan(y/x) but sigts of x and y are used to determine the location of the result

y (NUMERIC) - Input Y variable
x (NUMERIC) - Input X variable

atanh

Elementwise atanh (inverse hyperbolic tangent) function: out = atanh(x)

x (NUMERIC) - Input variable

bitShift

Bit shift operation

x (NUMERIC) - input
shift (NUMERIC) - shift value

bitShiftRight

Right bit shift operation

x (NUMERIC) - Input tensor
shift (NUMERIC) - shift argument

bitShiftRotl

Cyclic bit shift operation

x (NUMERIC) - Input tensor
shift (NUMERIC) - shift argy=ument

bitShiftRotr

Cyclic right shift operation

x (NUMERIC) - Input tensor
shift (NUMERIC) - Shift argument

ceil

Element-wise ceiling function: out = ceil(x).

Rounds each value up to the nearest integer value (if not already an integer)

x (NUMERIC) - Input variable

clipByNorm

Clipping by L2 norm, optionally along dimension(s)

if l2Norm(x,dimension) < clipValue, then input is returned unmodifed

Otherwise, out[i] = in[i] * clipValue / l2Norm(in, dimensions) where each value is clipped according

to the corresponding l2Norm along the specified dimensions

x (NUMERIC) - Input variable
clipValue - Clipping value (maximum l2 norm)
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

clipByValue

Element-wise clipping function:

out[i] = in[i] if in[i] >= clipValueMin and in[i] <= clipValueMax

out[i] = clipValueMin if in[i] < clipValueMin

out[i] = clipValueMax if in[i] > clipValueMax

x (NUMERIC) - Input variable
clipValueMin - Minimum value for clipping
clipValueMax - Maximum value for clipping

confusionMatrix

Compute the 2d confusion matrix of size [numClasses, numClasses] from a pair of labels and predictions, both of

which are represented as integer values. This version assumes the number of classes is 1 + max(max(labels), max(pred))

For example, if labels = [0, 1, 1] and predicted = [0, 2, 1] then output is:

[1, 0, 0]

[0, 1, 1]

[0, 0, 0]

labels (NUMERIC) - Labels - 1D array of integer values representing label values
pred (NUMERIC) - Predictions - 1D array of integer values representing predictions. Same length as labels
dataType - Data type

confusionMatrix

Compute the 2d confusion matrix of size [numClasses, numClasses] from a pair of labels and predictions, both of

which are represented as integer values.

For example, if labels = [0, 1, 1], predicted = [0, 2, 1], and numClasses=4 then output is:

[1, 0, 0, 0]

[0, 1, 1, 0]

[0, 0, 0, 0]

labels (NUMERIC) - Labels - 1D array of integer values representing label values
pred (NUMERIC) - Predictions - 1D array of integer values representing predictions. Same length as labels
numClasses - Number of classes

confusionMatrix

Compute the 2d confusion matrix of size [numClasses, numClasses] from a pair of labels and predictions, both of

which are represented as integer values. This version assumes the number of classes is 1 + max(max(labels), max(pred))

For example, if labels = [0, 1, 1], predicted = [0, 2, 1] and weights = [1, 2, 3]

[1, 0, 0]

[0, 3, 2]

[0, 0, 0]

labels (NUMERIC) - Labels - 1D array of integer values representing label values
pred (NUMERIC) - Predictions - 1D array of integer values representing predictions. Same length as labels
weights (NUMERIC) - Weights - 1D array of values (may be real/decimal) representing the weight/contribution of each prediction. Must be same length as both labels and predictions arrays

confusionMatrix

Compute the 2d confusion matrix of size [numClasses, numClasses] from a pair of labels and predictions, both of

which are represented as integer values.

For example, if labels = [0, 1, 1], predicted = [0, 2, 1], numClasses = 4, and weights = [1, 2, 3]

[1, 0, 0, 0]

[0, 3, 2, 0]

[0, 0, 0, 0]

labels (NUMERIC) - Labels - 1D array of integer values representing label values
pred (NUMERIC) - Predictions - 1D array of integer values representing predictions. Same length as labels
weights (NUMERIC) - Weights - 1D array of values (may be real/decimal) representing the weight/contribution of each prediction. Must be same length as both labels and predictions arrays

cos

Elementwise cosine operation: out = cos(x)

x (NUMERIC) - Input variable

cosh

Elementwise cosh (hyperbolic cosine) operation: out = cosh(x)

x (NUMERIC) - Input variable

cosineDistance

Cosine distance reduction operation. The output contains the cosine distance for each

tensor/subset along the specified dimensions:

out = 1.0 - cosineSimilarity(x,y)

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensions - Dimensions to calculate cosineDistance over (Size: AtLeast(min=0))

cosineSimilarity

Cosine similarity pairwise reduction operation. The output contains the cosine similarity for each tensor/subset

along the specified dimensions:

out = (sum_i x[i] y[i]) / ( sqrt(sum_i x[i]^2) sqrt(sum_i y[i]^2)

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensions - Dimensions to calculate cosineSimilarity over (Size: AtLeast(min=0))

countNonZero

Count non zero array reduction operation, optionally along specified dimensions: out = count(x != 0)

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

countZero

Count zero array reduction operation, optionally along specified dimensions: out = count(x == 0)

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

cross

Returns the pair-wise cross product of equal size arrays a and b: a x b = ||a||x||b|| sin(theta).

Can take rank 1 or above inputs (of equal shapes), but note that the last dimension must have dimension 3

a (NUMERIC) - First input
b (NUMERIC) - Second input

cube

Element-wise cube function: out = x^3

x (NUMERIC) - Input variable

diag

Returns an output variable with diagonal values equal to the specified values; off-diagonal values will be set to 0

For example, if input = [1,2,3], then output is given by:

[ 1, 0, 0]

[ 0, 2, 0]

[ 0, 0, 3]

Higher input ranks are also supported: if input has shape [a,...,R-1] then output[i,...,k,i,...,k] = input[i,...,k].

i.e., for input rank R, output has rank 2R

x (NUMERIC) - Input variable

diagPart

Extract the diagonal part from the input array.

If input is

[ 1, 0, 0]

[ 0, 2, 0]

[ 0, 0, 3]

then output is [1, 2, 3].

Supports higher dimensions: in general, out[i,...,k] = in[i,...,k,i,...,k]

x (NUMERIC) - Input variable

div

Pairwise division operation, out = x / y

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

div

Scalar division operation, out = in / scalar

x (NUMERIC) - Input variable
value - Scalar value for op

entropy

Entropy reduction: -sum(x * log(x))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

erf

Element-wise Gaussian error function - out = erf(in)

x (NUMERIC) - Input variable

erfc

Element-wise complementary Gaussian error function - out = erfc(in) = 1 - erf(in)

x (NUMERIC) - Input variable

euclideanDistance

Euclidean distance (l2 norm, l2 distance) reduction operation. The output contains the Euclidean distance for each

tensor/subset along the specified dimensions:

out = sqrt( sum_i (x[i] - y[i])^2 )

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensions - Dimensions to calculate euclideanDistance over (Size: AtLeast(min=0))

exp

Elementwise exponent function: out = exp(x) = 2.71828...^x

x (NUMERIC) - Input variable

expm1

Elementwise 1.0 - exponent function: out = 1.0 - exp(x) = 1.0 - 2.71828...^x

x (NUMERIC) - Input variable

eye

Generate an identity matrix with the specified number of rows and columns.

rows - Number of rows

eye

As per eye(String, int, int, DataType) but with the default datatype, Eye.DEFAULT_DTYPE

rows - Number of rows
cols - Number of columns

eye

Generate an identity matrix with the specified number of rows and columns

Example:

rows - Number of rows
cols - Number of columns
dataType - Data type

eye

As per eye(int, int) bit with the number of rows/columns specified as scalar INDArrays

rows (INT) - Number of rows
cols (INT) - Number of columns

eye

As per eye(String, int) but with the number of rows specified as a scalar INDArray

rows (INT) - Number of rows

firstIndex

First index reduction operation.

Returns a variable that contains the index of the first element that matches the specified condition (for each

slice along the specified dimensions)

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

in (NUMERIC) - Input variable
condition - Condition to check on input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=1))

floor

Element-wise floor function: out = floor(x).

Rounds each value down to the nearest integer value (if not already an integer)

x (NUMERIC) - Input variable

floorDiv

Pairwise floor division operation, out = floor(x / y)

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

floorMod

Pairwise Modulus division operation

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

floorMod

Scalar floor modulus operation

x (NUMERIC) - Input variable
value - Scalar value for op

hammingDistance

Hamming distance reduction operation. The output contains the cosine distance for each

tensor/subset along the specified dimensions:

out = count( x[i] != y[i] )

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensions - Dimensions to calculate hammingDistance over (Size: AtLeast(min=0))

iamax

Index of the max absolute value: argmax(abs(in))

see argmax(String, INDArray, boolean, int...)

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=1))
keepDims - If true: keep the dimensions that are reduced on (as length 1). False: remove the reduction dimensions - default = false

iamin

Index of the min absolute value: argmin(abs(in))

see argmin(String, INDArray, boolean, int...)

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=1))
keepDims - If true: keep the dimensions that are reduced on (as length 1). False: remove the reduction dimensions - default = false

isFinite

Is finite operation: elementwise isFinite(x)

Returns an array with the same shape/size as the input, with values 1 where condition is satisfied, or

value 0 otherwise

x (NUMERIC) - Input variable

isInfinite

Is infinite operation: elementwise isInfinite(x)

Returns an array with the same shape/size as the input, with values 1 where condition is satisfied, or

value 0 otherwise

x (NUMERIC) - Input variable

isMax

Is maximum operation: elementwise x == max(x)

Returns an array with the same shape/size as the input, with values 1 where condition is satisfied, or

value 0 otherwise

x (NUMERIC) - Input variable

isNaN

Is Not a Number operation: elementwise isNaN(x)

Returns an array with the same shape/size as the input, with values 1 where condition is satisfied, or

value 0 otherwise

x (NUMERIC) - Input variable

isNonDecreasing

Is the array non decreasing?

An array is non-decreasing if for every valid i, x[i] <= x[i+1]. For Rank 2+ arrays, values are compared

in 'c' (row major) order

x (NUMERIC) - Input variable

isStrictlyIncreasing

Is the array strictly increasing?

An array is strictly increasing if for every valid i, x[i] < x[i+1]. For Rank 2+ arrays, values are compared

in 'c' (row major) order

x (NUMERIC) - Input variable

jaccardDistance

Jaccard similarity reduction operation. The output contains the Jaccard distance for each

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensions - Dimensions to calculate jaccardDistance over (Size: AtLeast(min=0))

lastIndex

Last index reduction operation.

Returns a variable that contains the index of the last element that matches the specified condition (for each

slice along the specified dimensions)

Note that if keepDims = true, the output variable has the same rank as the input variable,

with the reduced dimensions having size 1. This can be useful for later broadcast operations (such as subtracting

the mean along a dimension).

Example: if input has shape [a,b,c] and dimensions=[1] then output has shape:

keepDims = true: [a,1,c]

keepDims = false: [a,c]

in (NUMERIC) - Input variable
condition - Condition to check on input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=1))

listDiff

Calculates difference between inputs X and Y.

x (NUMERIC) - Input variable X
y (NUMERIC) - Input variable Y

log

Element-wise logarithm function (base e - natural logarithm): out = log(x)

x (NUMERIC) - Input variable

log

Element-wise logarithm function (with specified base): out = log_{base`(x)

x (NUMERIC) - Input variable
base - Logarithm base

log1p

Elementwise natural logarithm function: out = log_e (1 + x)

x (NUMERIC) - Input variable

logEntropy

Log entropy reduction: log(-sum(x * log(x)))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

logSumExp

Log-sum-exp reduction (optionally along dimension).

Computes log(sum(exp(x))

input (NUMERIC) - Input variable
dimensions - Optional dimensions to reduce along (Size: AtLeast(min=0))

manhattanDistance

Manhattan distance (l1 norm, l1 distance) reduction operation. The output contains the Manhattan distance for each

tensor/subset along the specified dimensions:

out = sum_i abs(x[i]-y[i])

x (NUMERIC) - Input variable x
y (NUMERIC) - Input variable y
dimensions - Dimensions to calculate manhattanDistance over (Size: AtLeast(min=0))

matrixDeterminant

Matrix determinant op. For 2D input, this returns the standard matrix determinant.

For higher dimensional input with shape [..., m, m] the matrix determinant is returned for each

shape [m,m] sub-matrix.

in (NUMERIC) - Input

matrixInverse

Matrix inverse op. For 2D input, this returns the standard matrix inverse.

For higher dimensional input with shape [..., m, m] the matrix inverse is returned for each

shape [m,m] sub-matrix.

in (NUMERIC) - Input

max

Pairwise max operation, out = max(x, y)

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - First input variable, x
y (NUMERIC) - Second input variable, y

mergeAdd

Merge add function: merges an arbitrary number of equal shaped arrays using element-wise addition:

out = sum_i in[i]

inputs (NUMERIC) - Input variables

mergeAvg

Merge average function: merges an arbitrary number of equal shaped arrays using element-wise mean operation:

out = mean_i in[i]

inputs (NUMERIC) - Input variables

mergeMax

Merge max function: merges an arbitrary number of equal shaped arrays using element-wise maximum operation:

out = max_i in[i]

inputs (NUMERIC) - Input variables

meshgrid

Broadcasts parameters for evaluation on an N-D grid.

inputs (NUMERIC) -
cartesian -

min

Pairwise max operation, out = min(x, y)

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - First input variable, x
y (NUMERIC) - Second input variable, y

mod

Pairwise modulus (remainder) operation, out = x % y

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

moments

Calculate the mean and (population) variance for the input variable, for the specified axis

input (NUMERIC) - Input to calculate moments for
axes - Dimensions to perform calculation over (Size: AtLeast(min=0))

mul

Pairwise multiplication operation, out = x * y

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

mul

Scalar multiplication operation, out = in * scalar

x (NUMERIC) - Input variable
value - Scalar value for op

neg

Elementwise negative operation: out = -x

x (NUMERIC) - Input variable

normalizeMoments

Calculate the mean and variance from the sufficient statistics

counts (NUMERIC) - Rank 0 (scalar) value with the total number of values used to calculate the sufficient statistics
means (NUMERIC) - Mean-value sufficient statistics: this is the SUM of all data values
variances (NUMERIC) - Variaance sufficient statistics: this is the squared sum of all data values

or

Boolean OR operation: elementwise (x != 0) || (y != 0)

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

Returns an array with values 1 where condition is satisfied, or value 0 otherwise.

x (BOOL) - Input 1
y (BOOL) - Input 2

pow

Element-wise power function: out = x^value

x (NUMERIC) - Input variable
value - Scalar value for op

pow

Element-wise (broadcastable) power function: out = x[i]^y[i]

x (NUMERIC) - Input variable
y (NUMERIC) - Power

rationalTanh

Rational Tanh Approximation elementwise function, as described in the paper:

Compact Convolutional Neural Network Cascade for Face Detection

This is a faster Tanh approximation

x (NUMERIC) - Input variable

rdiv

Pairwise reverse division operation, out = y / x

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

rdiv

Scalar reverse division operation, out = scalar / in

x (NUMERIC) - Input variable
value - Scalar value for op

reciprocal

Element-wise reciprocal (inverse) function: out[i] = 1 / in[i]

x (NUMERIC) - Input variable

rectifiedTanh

Rectified tanh operation: max(0, tanh(in))

x (NUMERIC) - Input variable

round

Element-wise round function: out = round(x).

Rounds (up or down depending on value) to the nearest integer value.

x (NUMERIC) - Input variable

rsqrt

Element-wise reciprocal (inverse) of square root: out = 1.0 / sqrt(x)

x (NUMERIC) - Input variable

rsub

Pairwise reverse subtraction operation, out = y - x

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

rsub

Scalar reverse subtraction operation, out = scalar - in

x (NUMERIC) - Input variable
value - Scalar value for op

setDiag

Set the diagonal value to the specified values

If input is

[ a, b, c]

[ d, e, f]

[ g, h, i]

and diag = [ 1, 2, 3] then output is

[ 1, b, c]

[ d, 2, f]

[ g, h, 3]

in (NUMERIC) - Input variable
diag (NUMERIC) - Diagonal

shannonEntropy

Shannon Entropy reduction: -sum(x * log2(x))

in (NUMERIC) - Input variable
dimensions - Dimensions to reduce over. If dimensions are not specified, full array reduction is performed (Size: AtLeast(min=0))

sign

Element-wise sign (signum) function:

out = -1 if in < 0

out = 0 if in = 0

out = 1 if in > 0

x (NUMERIC) - Input variable

sin

Elementwise sine operation: out = sin(x)

x (NUMERIC) - Input variable

sinh

Elementwise sinh (hyperbolic sine) operation: out = sinh(x)

x (NUMERIC) - Input variable

sqrt

Element-wise square root function: out = sqrt(x)

x (NUMERIC) - Input variable

square

Element-wise square function: out = x^2

x (NUMERIC) - Input variable

squaredDifference

Pairwise squared difference operation.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

standardize

Standardize input variable along given axis

out = (x - mean) / stdev

with mean and stdev being calculated along the given dimension.

For example: given x as a mini batch of the shape [numExamples, exampleLength]:

use dimension 1 too use the statistics (mean, stdev) for each example
use dimension 0 if you want to use the statistics for each column across all examples
use dimensions 0,1 if you want to use the statistics across all columns and examples

step

Elementwise step function:

out(x) = 1 if x >= cutoff

out(x) = 0 otherwise

x (NUMERIC) - Input variable
value - Scalar value for op

sub

Pairwise subtraction operation, out = x - y

Note: supports broadcasting if x and y have different shapes and are broadcastable.

For example, if X has shape [1,10] and Y has shape [5,10] then op(X,Y) has output shape [5,10]

Broadcast rules are the same as NumPy:

x (NUMERIC) - Input variable
y (NUMERIC) - Input variable

sub

Scalar subtraction operation, out = in - scalar

x (NUMERIC) - Input variable
value - Scalar value for op

tan

Elementwise tangent operation: out = tan(x)

x (NUMERIC) - Input variable

tanh

Elementwise tanh (hyperbolic tangent) operation: out = tanh(x)

x (NUMERIC) - Input variable

trace

Matrix trace operation

For rank 2 matrices, the output is a scalar vith the trace - i.e., sum of the main diagonal.

For higher rank inputs, output[a,b,c] = trace(in[a,b,c,:,:])

in (NUMERIC) - Input variable

xor

Boolean XOR (exclusive OR) operation: elementwise (x != 0) XOR (y != 0)

If x and y arrays have equal shape, the output shape is the same as these inputs.

Note: supports broadcasting if x and y have different shapes and are broadcastable.

Returns an array with values 1 where condition is satisfied, or value 0 otherwise.

x (BOOL) - Input 1
y (BOOL) - Input 2

zeroFraction

Full array zero fraction array reduction operation, optionally along specified dimensions: out = (count(x == 0) / length(x))

input (NUMERIC) - Input variable

Reference

Operation Namespaces

hashtagNamespaces

Bitwise

hashtagand

Image

hashtagCropAndResize

hashtagadjustContrast

hashtagadjustHue

hashtagadjustSaturation

hashtagextractImagePatches

hashtaghsvToRgb

hashtagimageResize

hashtagnonMaxSuppression

hashtagrandomCrop

hashtagrgbToHsv

hashtagrgbToYiq

hashtagrgbToYuv

hashtagyiqToRgb

hashtagyuvToRgb

LinAlg

hashtagCholesky

hashtagLstsq

hashtagLu

hashtagMatmul

hashtagMatrixBandPart

hashtagQr

hashtagSolve

hashtagTriangularSolve

hashtagcross

hashtagdiag

hashtagdiag_part

hashtaglogdet

hashtagmmul

hashtagsvd

hashtagtri

hashtagtriu

Loss

hashtagabsoluteDifference

hashtagcosineDistance

hashtaghingeLoss

hashtaghuberLoss

hashtagl2Loss

hashtaglogLoss

hashtaglogPoisson

hashtagmeanPairwiseSquaredError

hashtagmeanSquaredError

hashtagsigmoidCrossEntropy

hashtagsoftmaxCrossEntropy

hashtagsparseSoftmaxCrossEntropy

hashtagweightedCrossEntropyWithLogits

Random

hashtagbernoulli

hashtagbinomial

hashtagexponential

hashtaglogNormal

hashtagnormal

hashtagnormalTruncated

hashtaguniform

RNN

hashtagOperation classes

hashtaggru

Operation Namespaces

hashtagNamespaces

Image

hashtagCropAndResize

hashtagadjustContrast

hashtagadjustHue

hashtagadjustSaturation

hashtagextractImagePatches

hashtaghsvToRgb

hashtagimageResize

hashtagnonMaxSuppression

hashtagrandomCrop

hashtagrgbToHsv

hashtagrgbToYiq

hashtagrgbToYuv

hashtagyiqToRgb

hashtagyuvToRgb

RNN

Namespaces

and

CropAndResize

adjustContrast

adjustHue

adjustSaturation

extractImagePatches

hsvToRgb

imageResize

nonMaxSuppression

randomCrop

rgbToHsv

rgbToYiq

rgbToYuv

yiqToRgb

yuvToRgb

Cholesky

Lstsq

Lu

Matmul

MatrixBandPart

Qr

Solve

TriangularSolve

cross

diag

diag_part

logdet

mmul

svd

tri

triu

absoluteDifference

cosineDistance

hingeLoss

huberLoss

l2Loss

logLoss

logPoisson

meanPairwiseSquaredError

meanSquaredError

sigmoidCrossEntropy

softmaxCrossEntropy

sparseSoftmaxCrossEntropy

weightedCrossEntropyWithLogits

bernoulli

binomial

exponential

logNormal

normal

normalTruncated

uniform

Operation classes

gru

Namespaces

CropAndResize

adjustContrast

adjustHue

adjustSaturation

extractImagePatches

hsvToRgb

imageResize

nonMaxSuppression

randomCrop

rgbToHsv

rgbToYiq

rgbToYuv

yiqToRgb

yuvToRgb

Operation classes

gru

gruCell

lstmCell

lstmLayer

lstmblock

sru

sruCell

Configuration Classes

LSTMConfiguration

LSTMLayerConfig