Running a GPU kernel

Forms that compile (18)

Syntax	Description	Note
Gpu.Load(current, source)	Loading an image into a buffer	It is scaled to fit when the sizes differ. When you use an image as numbers, turn sRGB off in that image’s import settings.
Gpu.Buffer(64, 64)	Creating a buffer	Once created, the size is also readable from the Udon side as buffer.width / buffer.height.
Gpu.Buffer(64, 64, GpuFormat.Half)	Creating with a chosen format	It can hold values outside 0..1 (negative values, values above 1), and the steps are finer. The area it takes is twice as large. The format has to be written out at the call (it cannot be passed in a variable). Gpu.Pack16x2 is a way of packing values into 8-bit components, so it is not used with this format. There is an upper limit to what it can hold: a value above 65504 stays at 65504. It does not become infinity, and no error or warning is raised. When you put values into this buffer, the side you put them from has to be a 16-bit float too. Loading from an 8-bit image with Gpu.Load flattens values outside 0..1 before they arrive. Nothing tells you they were flattened.
Gpu.Swap(ref a, ref b)	Swapping the two
Gpu.Run(nameof(Step), next, current)	Calling Run	The third argument onward map in order to the kernel’s second argument onward.
Gpu.Run(nameof(Step), next, current, phase)	Running with extra arguments
Gpu.Run("Step", next, current)	Naming a kernel with a string	Using nameof lets C# catch a misspelling.
Gpu.Run(nameof(Fade), current, next)	Multiple kernels
Gpu.Run(nameof(Blend), next, current, weights)	Running with a table	大きさは引数に [Capacity(16)] のように書きます（シェーダの中に定数として置かれるため、大きさが決まっていないと作れません）。大きさは 2 の冪で、上限は 1024 です。添字がその範囲を出たときは端から回り込みます。表はカーネルの中では読むだけです。1 つのカーネルに何枚でも取れますが、大きさの合計は 4000 までです（表はシェーダの定数として置かれます。定数は全体で 4096 までで、表以外の引数もそこに入ります）。渡した配列が無いときと、長さが書いた数と違うときは、その実行は何も起きません（何枚か取るときは、1 枚でも当てはまれば起きません）。
[Capacity(4)] Vector4[] w	Small table
Gpu.Run(nameof(Seed), board, 0.25f)	Running without a source buffer	The coordinates of the cell in charge come from the destination, so anything that does not read the previous frame (writing an initial state, a pattern made from coordinates alone) can be written this way.
Gpu.Reduce(nameof(Brighter), middle, full)	Specifying how to combine	One cell of the destination receives the range of the source that the cell is in charge of, folded into a single value. You line up the steps yourself: the number of times you write 64 → 8 → 1 is the number of runs.
Gpu.Max(peak, full)	Per-component maximum and minimum	The four components are folded independently. Give each call its own destination buffer: running Gpu.Max and Gpu.Min into the same one makes the second call overwrite the first result. When you need the color of the cell whose component is largest, write it with Gpu.Reduce.
Gpu.Show(current, display)	Displaying it on a Renderer
Gpu.Texture(current)	Retrieving it as a texture	What comes back is the buffer itself, not a copy.
VRCGraphics.Blit(Gpu.Texture(current), target)	Writing out to your own RenderTexture	Create the destination with RenderTextureReadWrite.Linear (the default adds sRGB, so the value the kernel returned does not arrive unchanged). Setting FilterMode to Point makes it read the same way as a buffer from Gpu.Buffer.
public GpuBuffer2D target;	Plugging the destination in from outside	Plug a RenderTexture of your own into it and that becomes the destination the kernel writes to. Its own settings are used as they are, so match sRGB and FilterMode the same way as above.
VRCAsyncGPUReadback.Request(Gpu.Texture(current), ...)	Reading a value back	It does not return on the spot: OnAsyncGpuReadbackComplete is called once the GPU has finished. From a default buffer, what arrives is bytes in the range 0..255, four per cell in the order R, G, B, A, and the value the kernel returned is that divided by 255. A buffer made with GpuFormat.Half has eight bytes per cell, so match the size of what you read into and the format you ask for: asking with TextureFormat.RGBAFloat lets you receive it as Color[]. When they do not match, no error and no warning appear and the back of what you read into stays zero. Ask for no more than you need, because bringing a whole surface back every frame spends what the GPU saved. Asking again before the answer arrives does not make it come sooner, and a late answer carries nothing that says which request it belongs to. Leave a buffer you have asked to read back alone until the answer arrives. Overwriting it while the request is in flight changes what comes back. It is not an error. With a front/back pair, two runs bring the same buffer back round.

Forms that don’t compile (18)

Syntax	Description	Error	Reason	Alternative
The kernel isn’t static	A method with [Kernel] that’s an instance method	TUKI0001	by design	Make it static
The return type isn’t Color4	The kernel returns a different type	TUKI0001	by design	Return Color4
The first argument isn’t KernelId	The kernel’s argument order is wrong	TUKI0001	by design	Make the first argument KernelId
The argument count for Gpu.Run doesn’t match	The count differs from the kernel’s arguments	TUKI0001	by design	Match the count
The table has no size	A Vector4[] parameter without [Capacity(N)]	TUKI0001	by design	Add [Capacity(N)]
The table size is not a power of two	A number such as [Capacity(20)]	TUKI0001	by design	Use a power of two
The table size exceeds 1024	A number such as [Capacity(2048)]	TUKI0001	by design	Use 1024 or less
Writing to the table	Assigning to a table element inside the kernel	TUKI0001	by design	Only read from it
The argument types for Gpu.Run don’t match	The types differ from the kernel’s arguments	TUKI0001	by design	Match the types
Gpu.Run(k, next, next)	The output buffer is also passed as an input	TUKI0001	by design	Pass a different buffer
Gpu.Run(warm ? nameof(A) : nameof(B), ...)	Choosing the kernel name at run time (a conditional, an array element or a field)	TUKI0001	by design	Branch, and call Gpu.Run in each branch
Gpu.Run("NotAKernel", ...)	Passing the name of a method without [Kernel]	TUKI0001	by design	Fix the spelling with nameof
Gpu.Run(nameof(fold), ...)	Passing a method marked [Reduce] to Gpu.Run	TUKI0001	by design	Run it with Gpu.Reduce
Gpu.Reduce(nameof(kernel), ...)	Passing a method marked [Kernel] to Gpu.Reduce	TUKI0001	by design	Run it with Gpu.Run
Gpu.Reduce(f, full, full)	Passing the same buffer as both the source and the destination	TUKI0001	by design	Fold into a different buffer
Calling Gpu.Pack16x2 outside a kernel	The packing is written outside the kernel	TUKI0001, TUKI0099, TUKI0102	by design	Pack it inside the kernel
Reading a buffer cell outside a kernel	The folded value is being pulled into an Udon-side variable	TUKI0099, TUKI0101, TUKI0102	by design	Pass it to the next kernel, or show it with Gpu.Show
Calling the kernel directly	Calling a method with [Kernel] as-is	TUKI0001, TUKI0099, TUKI0102	by design	Start it with Gpu.Run(nameof(…), …)

There is no way to assign a cell’s value straight into an Udon variable (the same holds for what Gpu.Reduce and Gpu.Max fold together); to use it as a number, take the buffer out with Gpu.Texture and hand it to the runtime’s asynchronous readback.

Buffers

Loading an image into a buffer

A still image, a video, or a camera’s output all work.

using UnityEngine;
using Tsukimi;

public class ParHostLoad : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;
    public Texture source;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
        Gpu.Load(current, source);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
    }
}

Creating a buffer

Each cell has 4 components at 8 bits each, and the contents start at zero.

using UnityEngine;
using Tsukimi;

public class ParHostBuffer : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
    }
}

Creating with a chosen format

Makes each component a 16-bit floating point number.

using UnityEngine;
using Tsukimi;

public class ParHostBufferFormat : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    void Start()
    {
        // create it with 16-bit floating point components, so values outside 0..1 fit too.
        // the format has to be known where you write it: you cannot pass a variable.
        current = Gpu.Buffer(64, 64, GpuFormat.Half);
        next = Gpu.Buffer(64, 64, GpuFormat.Half);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        // values an 8-bit buffer would clamp to 0..1 are kept as they are.
        Color4 c = prev[id];
        return new Color4(c.R * 2f - 0.5f, c.G, c.B, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
    }
}

Swapping the two

Use it when the previous result becomes the next input.

using UnityEngine;
using Tsukimi;

public class ParHostSwap : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
    }
}

Running

Calling Run

The second argument is the destination.

using UnityEngine;
using Tsukimi;

public class ParHostRun : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
    }
}

Running with extra arguments

The count and types of the values you pass are checked against the kernel’s arguments at compile time.

using UnityEngine;
using Tsukimi;

public class ParHostRunExtraArgs : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;
    private float phase;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev, float t)
    {
        return new Color4(prev[id].R * t, 0f, 0f, 1f);
    }

    void Update()
    {
        phase = phase + Time.deltaTime;
        Gpu.Run(nameof(Step), next, current, phase);
    }
}

Naming a kernel with a string

The kernel name can be passed as a string too.

using UnityEngine;
using Tsukimi;

public class ParHostRunStringName : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run("Step", next, current);
    }
}

Multiple kernels

Several kernels can be written and run in order.

using UnityEngine;
using Tsukimi;

public class ParHostTwoKernels : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    [Kernel]
    static Color4 Fade(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R * 0.5f, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Run(nameof(Fade), current, next);
    }
}

Running with a table

Pass an array of Vector4 and the kernel gets a table it can read by index.

using UnityEngine;
using Tsukimi;

public class ParHostRunTable : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    // Fill in 16 entries in the Inspector (the run does nothing unless the length is 16).
    public Vector4[] weights;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Blend(KernelId id, GpuBuffer2D prev, [Capacity(16)] Vector4[] w)
    {
        Color4 acc = Color4.Black;
        for (int i = 0; i < 16; i++)
        {
            acc = acc + prev[id.Offset(i - 8, 0)] * w[i].x;
        }
        return new Color4(Mathf.Clamp01(acc.R), Mathf.Clamp01(acc.G), Mathf.Clamp01(acc.B), 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Blend), next, current, weights);
        Gpu.Swap(ref current, ref next);
    }
}

Small table

Any power of two from 1 to 1024 can be written as the size.

using UnityEngine;
using Tsukimi;

public class ParHostRunSmallTable : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;

    // Any power of two from 1 to 1024 works as the size (the run does nothing unless the length is 4).
    public Vector4[] weights;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Blend(KernelId id, GpuBuffer2D prev, [Capacity(4)] Vector4[] w)
    {
        Color4 c = prev[id];
        // An index of 4 or more wraps around (w[4] is w[0])
        float v = c.R * w[0].x + c.G * w[1].x + c.B * w[2].x + c.A * w[3].x;
        return new Color4(v, v, v, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Blend), next, current, weights);
        Gpu.Swap(ref current, ref next);
    }
}

Running without a source buffer

A kernel does not have to take any buffer at all.

using UnityEngine;
using Tsukimi;

public class ParHostRunNoBuffer : TsukimiBehaviour
{
    private GpuBuffer2D board;

    void Start()
    {
        board = Gpu.Buffer(64, 64);
        Gpu.Run(nameof(Seed), board, 0.25f);
    }

    [Kernel]
    static Color4 Seed(KernelId id, float scale)
    {
        float u = (float)id.X / Gpu.OutWidth;
        return new Color4(u * scale, 0f, 0f, 1f);
    }
}

Reduction

Specifying how to combine

A method marked [Reduce] is the fold: it takes two cell values and returns one.

using UnityEngine;
using Tsukimi;

public class ParHostReduce : TsukimiBehaviour
{
    private GpuBuffer2D full;
    private GpuBuffer2D middle;
    private GpuBuffer2D one;

    void Start()
    {
        full = Gpu.Buffer(64, 64);
        middle = Gpu.Buffer(8, 8);
        one = Gpu.Buffer(1, 1);
    }

    [Reduce]
    static Color4 Brighter(Color4 a, Color4 b)
    {
        if (a.R != b.R) return a.R > b.R ? a : b;
        if (a.G != b.G) return a.G > b.G ? a : b;
        return a.B > b.B ? a : b;
    }

    public void Find()
    {
        Gpu.Reduce(nameof(Brighter), middle, full);
        Gpu.Reduce(nameof(Brighter), one, middle);
    }
}

Per-component maximum and minimum

A shortcut for when you don’t want to write a fold.

using UnityEngine;
using Tsukimi;

public class ParHostMax : TsukimiBehaviour
{
    public Renderer display;
    private GpuBuffer2D full;
    private GpuBuffer2D next;
    private GpuBuffer2D peak;
    private GpuBuffer2D floor;

    [Kernel]
    static Color4 Stretch(KernelId id, GpuBuffer2D src, GpuBuffer2D peak, GpuBuffer2D floor)
    {
        Color4 hi = peak[new KernelId(0, 0)];
        Color4 lo = floor[new KernelId(0, 0)];
        float span = hi.R - lo.R;
        if (span < 0.0001f) return src[id];
        return (src[id] - lo) / span;
    }

    void Start()
    {
        full = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
        peak = Gpu.Buffer(1, 1);
        floor = Gpu.Buffer(1, 1);
    }

    public void Rebalance()
    {
        Gpu.Max(peak, full);
        Gpu.Min(floor, full);
        Gpu.Run(nameof(Stretch), next, full, peak, floor);
        Gpu.Show(next, display);
    }
}

Display and retrieval

Displaying it on a Renderer

Calling Gpu.Show replaces the target material’s main texture with the buffer (the material becomes a per-target copy).

using UnityEngine;
using Tsukimi;

public class ParHostShow : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;
    public Renderer display;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
        Gpu.Show(current, display);
    }
}

Retrieving it as a texture

Gpu.Texture is for passing to Unity or VRChat APIs.

using UnityEngine;
using Tsukimi;

public class ParHostTexture : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;
    private RenderTexture tex;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        tex = Gpu.Texture(current);
    }
}

Writing out to your own RenderTexture

The buffer you take out can be copied into a RenderTexture you prepared yourself.

using UnityEngine;
using VRC.SDKBase;
using Tsukimi;

public class ParHostBlitOwnRt : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;
    public RenderTexture target;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
        VRCGraphics.Blit(Gpu.Texture(current), target);
    }
}

Plugging the destination in from outside

A public GpuBuffer2D field shows up as a RenderTexture slot in the Inspector.

using UnityEngine;
using Tsukimi;

public class ParHostBufferSlot : TsukimiBehaviour
{
    public GpuBuffer2D target;
    private GpuBuffer2D source;

    void Start()
    {
        source = Gpu.Buffer(64, 64);
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        Gpu.Run(nameof(Step), target, source);
    }
}

Reading a value back

To receive what is in a buffer as numbers, ask for a readback.

using UnityEngine;
using VRC.SDK3.Rendering;
using VRC.Udon.Common.Interfaces;
using Tsukimi;

public class ParHostReadback : TsukimiBehaviour
{
    private GpuBuffer2D current;
    private GpuBuffer2D next;
    private byte[] cell;
    private bool waiting;
    public float level;

    void Start()
    {
        current = Gpu.Buffer(64, 64);
        next = Gpu.Buffer(64, 64);
        cell = new byte[4];   // one cell is four bytes
    }

    [Kernel]
    static Color4 Step(KernelId id, GpuBuffer2D prev)
    {
        return new Color4(prev[id].R, 0f, 0f, 1f);
    }

    void Update()
    {
        // we do not run again until what we asked for has arrived.
        // a buffer we asked to read back must not be overwritten until the answer arrives.
        // with a front/back pair, two runs bring the same buffer back round.
        if (waiting) return;

        Gpu.Run(nameof(Step), next, current);
        Gpu.Swap(ref current, ref next);
        waiting = true;

        // ask for the single cell at x = 32, y = 32
        VRCAsyncGPUReadback.Request(
            Gpu.Texture(current), 0, 32, 1, 32, 1, 0, 1,
            TextureFormat.RGBA32, (IUdonEventReceiver)this);
    }

    public override void OnAsyncGpuReadbackComplete(VRCAsyncGPUReadbackRequest request)
    {
        waiting = false;
        if (request.hasError) return;
        if (!request.TryGetData(cell, 0)) return;

        // arrives as 0..255, in the order R, G, B, A
        level = cell[0] / 255f;
    }
}