SameDiffinstance - be it weights, bias, inputs, activations or general parameters - all are handled by objects of class
SameDiffbelong to one of four variable types, constituting an enumeration
VariableType. Here they are:
VARIABLE: are trainable parameters of your network, e.g. weights and bias of a layer. Naturally, we want them
CONSTANT: are those parameters which, like variables, are persistent for the network, but are not being
PLACEHOLDER: store temporary values that are to be supplied from the outside, like inputs and labels.
PLACEHOLDERis not persistent.
ARRAY: are temporary values as well, representing outputs of operations within a
PLACEHOLDER, are not persistent.
getVariableType, like so:
INDArraymay be obtained using
getArr(true)- the latter one if you wish the program to throw an exception if the variable's value is not initialized.
DataTypeenum. Currently in
DataTypethere are three floating point types:
HALF; four integer types:
UBYTE; one boolean type
BOOL- all of them will be referred as numeric types. In addition, there is a string type dubbed
UTF8; and two helper data types
UNKNOWN. The 16-bit floating point format
BFLOAT16and unsigned integer types (
UINT64) will be available in
weightsto be of one of the floating point data types, and will throw an exception otherwise. Also, as we shall discuss just below, all the
VARIABLEare supposed to be of floating point type.
SDVariable's involved in an operation are to belong to the same
SameDiff. As we briefly mentioned above, variables' values need to be both preserved for application, and updated during training. Training means, that we iteratively update the values by small fractions of their gradients, and this only makes sense if variables are of floating point types (see data types above).
SameDiffusing different versions of
varfunction from your
SameDiffinstance. For example, the code
floatnumbers - weights for a single layer MNIST perceptron in this case - to a pre-existing
1/28. You may put any other array creation methods instead of
nrand, or any preset array, of course. Also, you may use some popular initialization scheme, like so:
booleanare allowed alongside with
SameDiffby means of
constantmethods. A constant may be created form an
INDArray, like that:
SameDiffare inputs and, when applicable, labels. You may create placeholders of any data type, depending on the operations you use them in. To add a placeholder to a
SameDiff, you may call one of
placeHoldermethods, e.g. like that:
ARRAYtype appear as outputs of operations within
SameDiff. Accordingly, the data type of an array-type variable depends on the kind of operation it is produced by and variable type(s) ot its argument(s). Arrays are not persistent - they are one-time values that will be recalculated from scratch at the next step. However, unlike placeholders, gradients are computed for them, as those are needed to update the values of
someVariableis an instance of
VARIABLEtype. The variable
someVariablewill not be trained any more.
SameDiffhas its unique
SameDiffactually tracks your variables by their names, and allows you to retrieve them by using
subwe actually have implicitly introduced a variable (of type
ARRAY) that holds the result of the subtraction. By adding a name into the operations's argument, we've secured ourselves the possibility to retrieve the variable from elsewhere: say, if later you need to infer the difference between the labels and the prediction as a vector, you may just write:
SameDiffinstance is initialized elsewhere, and you still need to get hold of some of its variables - say, multiple outputs.
setVarNamerespectively. When renaming, note that variable's name is to remain unique within its
INDArrayusing the method
eval(). Note that for non-persistent variables, the value should first be set. For variables with gradients, the gradient's value may also be inferred using the method