ONNX models (experimental)

This chapter is experimental. A trained model is baked into the code at compile time instead of being loaded at run time (no loading code, and no file to ship alongside it).

Two attributes

Attributes	Where it runs	Result	Syntax
`[Onnx("…")]`	Udon	The result can be read back as numbers. Suited to small models	ONNX models on Udon (experimental)
`[OnnxGpu("…")]`	GPU	Much larger models fit, but the result lands in a buffer, so using it as numbers takes one readback step	ONNX models on the GPU (experimental)

Pick the former when you read the values and branch on them, as in gesture recognition, and the latter when the result goes straight to the screen, as in reworking an image.

Generated names

Model file name	Prefix that is added
`gesture.onnx`	`Gesture`
`hand_pose.onnx`	`HandPose`

Underscores and the extension are dropped.

How to bake one in

Put the exported .onnx in the same folder as the script that will carry the attribute
Mark the class partial and save it with the attribute alone
Return to Unity: the import runs and the generated file appears next to the script
Write the code that uses the generated names


Writing both at once	Writing steps 2 and 4 in one go is a compile error, because the generated names do not exist yet
When it is rejected	The file generated last time and its weight textures are left as they are, and a small file such as `Judge.Gesture.Failed.Model.cs` is written beside them. The single `#error` in it gives the reason and stops compilation, so the previous weights neither run nor get uploaded. Fixing the model or removing the attribute removes that file (a model that has never been generated has no earlier file, so the code that uses the generated names fails as well)
When the model is swapped	Step 3 runs again on its own, and the element-count constants take their new values
When asked about Safe Mode	If the generated file is missing (you wrote both at once, or it was deleted by hand), the project fails to compile when the editor starts and you are asked whether to enter Safe Mode. Choose Ignore: the file is generated once loading finishes, and the project compiles

The generated file


Name	The script name joined with the model name. Writing the attribute for `gesture.onnx` in `Judge.cs` gives `Judge.Gesture.Model.cs`
Where it is placed	The same folder as the script carrying the attribute
Where the model is looked for	The path in the attribute is looked for next to the script first, then in the project folder
Regeneration	It is regenerated when either the script or the model changes, and removed when the attribute is deleted. Do not edit it by hand.
Size	The weights become code as they are, so it runs to more lines than the number of weights, and all of it is read on every compile
Order of writing	Writing the attribute and the code that uses the generated names at the same time is a compile error. Save the attribute on its own first.

Supported operators


Operators	The operators that are accepted, and their conditions, are in ONNX model operators (experimental). A model containing an operator not listed there is an error at compile time
Layer order	The layers of a model do not have to be listed in the order they run (like the original runtime, the conversion puts each layer after the layers whose outputs it reads). Layers that read each other’s outputs in a loop cannot be put in such an order, so the model is an error that names those layers (the original runtime refuses such a model too)

Accuracy

The result is not guaranteed to match the original model. Values are rounded when quantized to 8 bits, and values carried in half precision follow how the GPU rounds when it stores them in half precision (toward zero, as measured on one PC graphics card), so they differ from what the exporting runtime computes by that much (for a floating-point model, by the rounding to 16 bits).