ONNX model operators (experimental)

This chapter is experimental. A model containing an operator not listed here is an error at compile time.

Operators for the Udon attribute

Operators	Condition
Gemm	A and B are 2-D. transA, transB, alpha and beta are handled. The element types are float32, float64, and from opset 9 int32, uint32, int64 and uint64. With integers, only alpha and beta of 1 are accepted
MatMul	A 1-D operand is not accepted. Leading axes are broadcast. The element types are the same as for Gemm
Add	The two inputs are broadcast. Both have the same element type. The element types are float32, float64 and the integers (32- and 64-bit from opset 6, 8- and 16-bit from opset 14). bool is not accepted
Sub	The two inputs are broadcast. The element types are the same as for Add
Mul	The two inputs are broadcast. The element types are the same as for Add
Relu	The element types are float32, float64 and, from opset 14, the signed integers (int8, int16, int32, int64). Unsigned integers and bool are not taken. NaN and -0 are returned as they are, as in the original runtime
Identity	No conditions. Every element type Udon can hold is accepted, including bool
Flatten	axis must be within the input’s rank. Element types other than float32 and float64 are accepted from opset 9
Reshape	The shape must be a constant (a shape passed at run time is not accepted). Element types other than float32 and float64 are accepted from opset 5

Operators for the GPU attribute (8-bit models)

Operators	Condition
Conv	kernel_shape has the same height and width. strides and dilations are supported. group is 1 or equal to the input and output channel count (depthwise) only. auto_pad can be SAME_UPPER, SAME_LOWER or VALID (SAME cannot be combined with dilations)
DepthToSpace	The layout is DCR or CRD. The block size is a power of 2, and the channel count must be divisible by its square
Concat	Only along the channel axis. Up to 4 inputs, each with a channel count that is a multiple of 4
Add	Both inputs come from earlier operators and have the same shape. Adding a constant is not accepted
QuantizeLinear	Read as the quantization scale, not as a computation. Scale and zero point are constants
DequantizeLinear	Read as the quantization scale, not as a computation. Putting a Cast (float16) and Cast (float32) pair in place of quantization carries that edge as half-precision floats
Pad	Only reflect, placed directly before a Conv with no padding of its own (it is folded into the Conv)
InstanceNormalization	scale and B are constants
Resize	Only linear interpolation. The coordinate_transformation_mode values half_pixel, pytorch_half_pixel, align_corners and asymmetric are handled. Only height and width can change
PRelu	The slope is a constant: either 1 value or a shape that applies along the channel axis (such as [C, 1, 1])
MaxPool	kernel_shape has the same height and width, and dilations is not accepted. auto_pad can be SAME_UPPER, SAME_LOWER or VALID. The positions of the maxima (Indices) cannot be output
Elu	8-bit input only
Relu	Half-precision floating-point inputs are taken too. NaN in the input can become 0 (“NaN” under Floating-point models)
LeakyRelu	Half-precision float input is accepted as well
Tanh	8-bit input only
Sigmoid	8-bit input only
Clip	The upper and lower bounds are constants. Half-precision floating-point inputs are taken too. When the lower bound is above the upper bound, every value becomes the upper bound, as in the original runtime. NaN in the input can become a bound
HardSigmoid	Handles alpha and beta
HardSwish	No conditions
Softplus	No conditions
Softsign	No conditions
Selu	Handles alpha and gamma
Celu	Handles alpha
Sin	No conditions
Exp	No conditions. In a floating-point model, one whose output can go past 65504 is named by a TUKI0135 warning
Neg	No conditions
Abs	No conditions
Cos	No conditions
Erf	No conditions
Atan	No conditions
Floor	No conditions. In a floating-point model the input is read rounded to 16 bits, so a value just below 0 or a negative integer gives an answer 1 larger than the original model’s
Asinh	No conditions
Round	Values exactly halfway are rounded to even. In a floating-point model the input is read rounded to 16 bits, so a value just past a half whose integer part is even (2.5001, for example) gives an answer 1 off from the original model’s (2, not 3) (from 1024 to 2048, for most values whose fraction is one half or more)
Sign	No conditions. In a floating-point model, a value too small for 16 bits (below about 6e-8) gives 0
Shrink	bias and lambd are supported. In a floating-point model the input is read rounded to 16 bits, so a value just outside lambd gives 0 instead of an answer whose size is close to lambd minus bias
ThresholdedRelu	Handles alpha. In a floating-point model the input is read rounded to 16 bits, so a value just above alpha gives 0 instead of a value close to alpha
Log	An input of 0 or below gives NaN or infinity, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)
Sqrt	A negative input gives NaN, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)
Reciprocal	An input of 0 gives infinity, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)
Ceil	No conditions. In a floating-point model the input is read rounded to 16 bits, so a value just above 0 or a positive integer gives an answer 1 smaller than the original model’s (from 1024 to 2048, every positive value that is not a whole number)
Tan	No conditions
Asin	An input outside -1 to 1 gives NaN, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)
Acos	An input outside -1 to 1 gives NaN, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)
Sinh	No conditions
Cosh	No conditions
Acosh	An input below 1 gives NaN, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)
Atanh	An input outside -1 to 1 gives NaN or infinity, as in the original model (in an 8-bit model, values for inputs outside the domain are not guaranteed to match the original model)

Operators for the GPU attribute (floating-point models only)

Operators	Condition
Split	Any axis can be split. The split lengths are constants (the split attribute is also read)
Sub	One side may be a constant, and it is broadcast (a larger constant, which makes the output larger than the earlier output, is also accepted). When both are outputs of earlier operators, each axis has the same length or one of them is 1 (when one has the output shape, the other’s axes stored as channels are all 1 or none; shapes that both broadcast, such as [1, 6, 1] and [1, 1, 8], are also accepted)
Mul	One side may be a constant, and it is broadcast (a larger constant, which makes the output larger than the earlier output, is also accepted). When both are outputs of earlier operators, each axis has the same length or one of them is 1 (when one has the output shape, the other’s axes stored as channels are all 1 or none; shapes that both broadcast, such as [1, 6, 1] and [1, 1, 8], are also accepted)
Div	When one side is a constant, it can be either the divisor or the dividend (an output larger than the output of the earlier operator is also accepted). When both are outputs of earlier operators, the conditions are the same as for Mul
Pow	One of the exponent and the base is a constant with a single value (a constant with more axes, which makes the output have more axes, is taken too). An output that can exceed 65504 is named by warning TUKI0135. A negative exponent that reads a value that can be 0 in 16 bits (such as a value with a constant too small for 16 bits added) is an error (Models that are accepted)
LayerNormalization	Only the last axis is normalized (when an image is reordered channels-last, the channels are normalized at each pixel). scale and bias are constants. The mean and the inverse standard deviation cannot be output
Softmax	Normalizes over the last axis (or that axis when all later axes have length 1), over all the axes that play the channel role, as with the channel axis of an image, or over the axis that plays the row role (T of [1, C, T], for example). The length need not be a multiple of 4
MatMul	When one side is a constant, it is held as 2-D weights (the constant may be on the left). Both may be outputs of earlier operators (the lengths need not be multiples of 4; a product of two values whose sizes are not known is named by a TUKI0135 warning). 8-bit and 4-bit weights are held as 16-bit values when the length is not a multiple of 4
Gemm	A and B are two-dimensional. transB, alpha and beta are supported, but transA is not. B can be the output of an earlier operator. C is a single value or one value per output column. A [1, K] input is taken as is
Gather	The form that looks up a vocabulary table along axis 0 (the ids are the model’s integer input, shape [1, T], 0 to 65535, and each row of the table holds a multiple of 4 values), and the form that picks from the output of an earlier operator with constant indices. Indices can be negative. Indices out of range, and forms that pick a different row for each row, are not accepted
RMSNormalization	Normalizes over the last axis only (the same conditions as LayerNormalization). scale is a constant
Swish	No conditions
GroupNormalization	The channel count must be divisible by the number of groups. scale and B are constants with one value per group. The Reshape → InstanceNormalization → Reshape pattern is also read as this operator (including the form that reshapes back to [1, C, H·W]). The layout of one-dimensional convolutions ([1, C, L]) is also supported
Slice	Any number of axes, with a step other than 0 (negative steps too). The range and steps are constants
LogSoftmax	Normalizes over the last axis (or that axis when all later axes have length 1), over all the axes that play the channel role, as with the channel axis of an image, or over the axis that plays the row role (T of [1, C, T], for example). The length need not be a multiple of 4
Hardmax	Normalizes over the last axis (or that axis when all later axes have length 1), over all the axes that play the channel role, as with the channel axis of an image, or over the axis that plays the row role (T of [1, C, T], for example). The length need not be a multiple of 4. When values tie, the first position is chosen. The first position is chosen even when the values differ only beyond 16 bits, so the position can differ from the original model’s
LpNormalization	Only the axis that plays the channel role is normalized (D of [1, T, D], axis 1 of an image [1, C, H, W]). p is 1 or 2. For an image, the channels of each pixel are normalized separately (the same when the channels are placed last)
Mod	One side is a constant with a single value, and fmod is 1. The input is read rounded to 16 bits, so a value just past a multiple of the divisor gives an answer that differs from the original model’s by about the divisor
Where	The condition is a constant (bool); of the other two inputs, one is a constant and the other is the output of an earlier operator. A constant or condition with a larger shape, which makes the output larger than the earlier output, is also accepted
Mish	No conditions
Gelu	approximate can be none or tanh
Max	Two or more outputs of earlier layers, and constants may be mixed in (nine or more inputs, and three or more with a constant mixed in, are split into a chain of layers that compare the inputs in order from the front). Outputs of earlier layers must have the same shape (with exactly two, a form where both broadcast to the output shape, such as [1, 6, 1] and [1, 1, 8], is also taken). With one output of an earlier layer and one constant, a form whose output is larger than the earlier output is taken too. NaN in the input can give the other value instead of NaN
Min	Two or more outputs of earlier layers, and constants may be mixed in (nine or more inputs, and three or more with a constant mixed in, are split into a chain of layers that compare the inputs in order from the front). Outputs of earlier layers must have the same shape (with exactly two, a form where both broadcast to the output shape, such as [1, 6, 1] and [1, 1, 8], is also taken). With one output of an earlier layer and one constant, a form whose output is larger than the earlier output is taken too. NaN in the input can give the other value instead of NaN
Sum	2 to 8 outputs of earlier operators (with the same shape), or one such output and one constant
Mean	2 to 8 outputs of earlier operators with the same shape. Constants cannot be mixed in
ReduceSum	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceMean	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceMax	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceMin	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceL1	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceL2	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceSumSquare	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported. One whose output can go past 65504 is named by a TUKI0135 warning
ReduceLogSumExp	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
ReduceProd	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported. One whose output can go past 65504 is named by a TUKI0135 warning
GlobalAveragePool	The input has 3 or more axes
GlobalMaxPool	The input has 3 or more axes
ReduceLogSum	The reduced axes are constants, and the axes stored as channels are either all included or none. keepdims is supported
AveragePool	A 1-D or 2-D window whose padding is smaller than the window. count_include_pad, auto_pad, ceil_mode and dilations are handled (a window that reads no input value gives 0)
SpaceToDepth	blocksize need not be a power of 2. mode must be DCR
ConvTranspose	One- or two-dimensional. group, dilations and auto_pad are also supported. output_shape is supported when it does not exceed the size that follows from the stride and the window. The weights are constants. The weights can be 8-bit or 4-bit (they are read back as 16-bit values, and a TUKI0123 warning names the layer)
BatchNormalization	Inference form only, with constant coefficients. It is folded into the weights of a directly preceding Conv
LpPool	p is 1 or 2. A 1-D or 2-D window whose padding is smaller than the window. auto_pad and ceil_mode are handled, and dilations are not accepted
MeanVarianceNormalization	Normalizes over the height and width of an image ([1, C, H, W]). In a layout whose only channel-role axis is the last one (D of [1, T, D], for example), normalizing over the last axis alone is also accepted. Axes of length 1 may appear in axes (the default [0, 2, 3] becomes the height-and-width form)
ArgMax	The axis the position is picked along has at most 2048 values, and the output can be used only as a model output (the position is held as an integer in a 16-bit value, so it cannot be passed to later operators). When that axis plays the channel role, it is the only axis that does. keepdims and select_last_index are handled. When the compared values become equal in 16 bits, the position can differ from the one the original model returns
ArgMin	The axis the position is picked along has at most 2048 values, and the output can be used only as a model output (the position is held as an integer in a 16-bit value, so it cannot be passed to later operators). When that axis plays the channel role, it is the only axis that does. keepdims and select_last_index are handled. When the compared values become equal in 16 bits, the position can differ from the one the original model returns
LSTM	The input has batch 1 with the time axis first (layout 0). Activations are the defaults, and clip, P (peepholes) and input_forget set to 1 are not accepted. The initial states (initial_h, initial_c) are absent or constant, and sequence_lens is accepted only as a constant whose values all equal the number of time steps. Bidirectional ones need a hidden_size that is a multiple of 4. The final states (Y_h, Y_c) can also be used
GRU	The input has batch 1 with the time axis first (layout 0). Activations are the defaults, linear_before_reset is 1 (the form PyTorch exports), and clip is not accepted. The initial state (initial_h) is absent or constant, and sequence_lens is accepted only as a constant whose values all equal the number of time steps. Bidirectional ones need a hidden_size that is a multiple of 4. The final state (Y_h) can also be used
RNN	The input has batch 1 with the time axis first (layout 0). The activation is Tanh (the default) or Relu in every direction, and clip is not accepted. The initial state (initial_h) is absent or constant, and sequence_lens is accepted only as a constant whose values all equal the number of time steps. Bidirectional ones need a hidden_size that is a multiple of 4. The final state (Y_h) can also be used

A floating-point model can also use the operators in the 8-bit table, and the conditions change for the following ones.

Operators	Condition in a floating-point model
Conv	Handles 1-D and 2-D, and the window may differ in height and width. group can be any number (the per-group weights become one set of weights padded with zeros). The input channel count need not be a multiple of 4. A Pad right before it (reflect, edge, or a constant that pads with 0) is read as the edge of the window. The weights are constants
Resize	nearest is also supported (reduction, non-integer scales, and all four nearest_mode values). cubic is supported when exclude_outside is 0. antialias of 1 (linear and cubic) is also supported. The opset 10 form (inputs X and scales) is also read. coordinate_transformation_mode also handles half_pixel_symmetric. linear and nearest on a 1-D signal ([1, C, L]) are also supported. keep_aspect_ratio_policy of not_larger and not_smaller is also handled when sizes is a constant and there is no roi (the output length is rounded to the nearest integer, and a length that differs from rounding down is accepted only when a 4-D input changes only its height and width)
InstanceNormalization	Only when axis 1 is the channel axis (a [1, T, D] input is an error)
PRelu	The slope is a single value or applies along the channel axis (D for [1, T, D], C for images)
DepthToSpace	The block size need not be a power of 2. The output channel count need not be a multiple of 4 (for example the RGB output of a super-resolution model)
Elu・Tanh・Sigmoid	Accepts floating-point input
Concat	Joins along the axis that plays the channel role (D for [1, T, D], C for an image). It can also join along the axis that plays the row role (T for [1, T, D], the height of an image) and the axis that plays the column role (the width of an image). 9 or more inputs are read as joins of 8 or fewer. Each input’s channels need not be a multiple of 4. The axes outside the joined axis need not be 1. Inputs whose values are laid out with different roles give an error. Joins that mix outputs of earlier operators with float32 constants are also accepted (a CLS token, for example; there may be two or more outputs of earlier operators)
MaxPool	1-D windows and windows whose height and width differ are also handled. dilations are also handled (one that has a window reading no input value gives an error)
MatMul	A product of an image reordered channels-last with weights is read as a product at each pixel (a 1×1 Conv)
Add	One side may be a constant (an output larger than the output of the earlier operator is also accepted). When both are outputs of earlier operators, the conditions are the same as for Sub
Pad	reflect, edge, constant (with a constant value) and, from opset 19, wrap can be used on any axis. Negative widths (cropping) are also supported. A form that cannot be folded into the Conv right after it is padded by copying values
DequantizeLinear	Read only as the weights of a model whose weights alone are in 8 or 4 bits
Reshape・Transpose	Accepts changes of the apparent shape that keep the order of the values, merging the rows and columns of an image into one axis (or splitting them), and a Transpose that reorders the axes playing the channel role. Flattening an image or a sequence as it is and passing it to a product (Gemm, MatMul) is also accepted

Operators rewritten when the model is read

Operators	Accepted form
Squeeze・Unsqueeze・Flatten・Identity	Accepted when they do not move values (the same condition as Reshape)
Expand・Tile	Accepted when the shape (shape or repeats) is a constant
Constant	Tensor and number values are read as constants. String values are not accepted
Dropout	Accepted in the inference form (no training_mode, or a constant false) when the mask output is not used
Upsample	The opset 7 to 9 form is read as a Resize that gives the same values
Shape・Size・ConstantOfShape・Range・Cast・EyeLike・HannWindow・HammingWindow・BlackmanWindow・GatherElements・ScatterND・ScatterElements・NonZero・OneHot・CastLike・CumSum・CumProd・Mod・Sign・Clip・Compress・Xor・IsNaN・IsInf・MelWeightMatrix	Calculations that need only constants, and calculations that start from a Shape whose value the shape information decides, are computed while reading and become constants (the shape calculations left in a model that was not simplified)
MatMulNBits from com.microsoft	Read as a MatMul that multiplies by 4-bit weights turned back by DequantizeLinear (the conditions are under Floating-point models)
Scatter	The form up to opset 10 is read as a ScatterElements that gives the same values
Trilu	Read as a Where that selects with a constant condition, when k is a constant
GlobalLpPool	Read as ReduceL1 or ReduceL2 over the spatial axes, when p is 1 or 2
CenterCropPad	A constant target shape is read as a Pad whose widths include negative values (cropping)
GatherElements・GatherND・ReverseSequence・Compress	When the indices (sequence_lens, condition) are constants and each value moves along the channel, row and column axes independently, the operator is read as a step that copies values. Compress needs an axis
ScatterND・ScatterElements	When the indices and the written values are constants and no index repeats, the operator is read as a Where that selects with a constant condition
CumSum	Summing along the last axis (the axis that plays the channel role) is read as a product with a constant of zeros and ones
Cast・CastLike	When both the input and the output are float, the operator is read as one that does nothing
ReduceSum and the other reductions	When noop_with_empty_axes is 1 and axes is empty, the reduction is read, depending on the operator, as one that does nothing, as Abs, as a square or as Log
axes of Resize	When axes is given, it is expanded to scales and sizes for every axis
DFT	Forward and inverse transforms of real and complex inputs (including the onesided inverse transform) are read as a product with constant weights. axis and dft_length are constants
Col2Im	A 1-D or 2-D overlap-add (PyTorch’s fold, for example) is read as a ConvTranspose, only when image_shape and block_shape are constants. One that takes a model input directly gives an error
STFT	A real-valued signal ([1, L, 1]) whose frame_step, window and frame_length are constants is read as a one-dimensional Conv as long as the window