GPU Buffer Operations

Utilities for transferring data between CPU and GPU.

Memory Model

WebGPU uses explicit memory transfers:

CPU Memory              GPU Memory
┌─────────┐            ┌──────────┐
│ByteArray│  ────────► │  Buffer  │  writeBuffer (CPU → GPU)
└─────────┘            └──────────┘
                            │
                            ▼
                       [Compute Shader]
                            │
                            ▼
┌─────────┐            ┌──────────┐
│ByteArray│  ◄──────── │  Buffer  │  readBuffer  (GPU → CPU)
└─────────┘            └──────────┘

Buffer Usage Flags

Buffers must be created with appropriate usage flags:

• copyDst: Can receive data from CPU
• copySrc: Can send data to CPU
• storage: Can be read/written by compute shaders
• uniform: Can be used as uniform buffer

Example:

-- For model weights (upload once)
usage := [.storage, .copyDst]

-- For output logits (download to CPU)
usage := [.storage, .copySrc]

-- For input tokens (upload + read in shader)
usage := [.storage, .copyDst]

Asynchronous Operations

WebGPU operations are asynchronous:

1. Submit compute commands → Returns immediately
2. Use queue.onSubmittedWorkDone() to wait
3. Map buffer → Wait for GPU → Read data → Unmap

References

• WebGPU Spec: https://www.w3.org/TR/webgpu/
• Buffer mapping: https://gpuweb.github.io/gpuweb/#buffer-mapping

Data Type Conversion

def Hesper.WebGPU.BufferOps.floatToBytes (f : Float) : IO ByteArray

Convert Float32 to bytes (little-endian)

@param f Float32 value @return 4 bytes representing the float

Equations

• Hesper.WebGPU.BufferOps.floatToBytes f = Hesper.Basic.floatToBytes f

def Hesper.WebGPU.BufferOps.float32BitsToFloat64 (bits : UInt32) : Float

Alias for backward compatibility

Equations

• Hesper.WebGPU.BufferOps.float32BitsToFloat64 = Hesper.Basic.float32BitsToFloat64

def Hesper.WebGPU.BufferOps.bytesToFloat (bytes : ByteArray) (offset : Nat) : Float

Convert bytes to Float32 (little-endian)

@param bytes ByteArray (must have at least offset+4 bytes) @param offset Start offset @return Float32 value

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.uint32ToBytes (n : UInt32) : ByteArray

Convert UInt32 to bytes (little-endian)

@param n UInt32 value @return 4 bytes

Equations

• Hesper.WebGPU.BufferOps.uint32ToBytes n = (((ByteArray.empty.push (n &&& 255).toUInt8).push (n >>> 8 &&& 255).toUInt8).push (n >>> 16 &&& 255).toUInt8).push (n >>> 24 &&& 255).toUInt8

def Hesper.WebGPU.BufferOps.bytesToUInt32 (bytes : ByteArray) (offset : Nat) : UInt32

Convert bytes to UInt32 (little-endian)

@param bytes ByteArray @param offset Start offset @return UInt32 value

Equations

• One or more equations did not get rendered due to their size.

Array Conversions

def Hesper.WebGPU.BufferOps.floatArrayToBytes (arr : Array Float) : IO ByteArray

Convert Array Float to ByteArray (Float32 format)

@param arr Array of floats @return Packed ByteArray

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.bytesToFloatArray (bytes : ByteArray) : Array Float

Convert ByteArray to Array Float

@param bytes ByteArray (must be multiple of 4) @return Array of Float32 values

Equations

• Hesper.WebGPU.BufferOps.bytesToFloatArray bytes = Hesper.WebGPU.BufferOps.bytesToFloatArray.loop bytes (bytes.size / 4) 0 #[]

@[irreducible]

def Hesper.WebGPU.BufferOps.bytesToFloatArray.loop (bytes : ByteArray) (numFloats idx : Nat) (acc : Array Float) : Array Float

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.tokenArrayToBytes (arr : Array Nat) : ByteArray

Convert Array Nat (token IDs) to ByteArray (UInt32 format)

@param arr Array of token IDs @return Packed ByteArray

Equations

• Hesper.WebGPU.BufferOps.tokenArrayToBytes arr = Array.foldl (fun (bytes : ByteArray) (n : Nat) => bytes ++ Hesper.WebGPU.BufferOps.uint32ToBytes n.toUInt32) ByteArray.empty arr

def Hesper.WebGPU.BufferOps.bytesToTokenArray (bytes : ByteArray) : Array Nat

Convert ByteArray to Array Nat (token IDs)

@param bytes ByteArray (must be multiple of 4) @return Array of token IDs

Equations

• Hesper.WebGPU.BufferOps.bytesToTokenArray bytes = Hesper.WebGPU.BufferOps.bytesToTokenArray.loop bytes (bytes.size / 4) 0 #[]

@[irreducible]

def Hesper.WebGPU.BufferOps.bytesToTokenArray.loop (bytes : ByteArray) (numTokens idx : Nat) (acc : Array Nat) : Array Nat

Equations

• One or more equations did not get rendered due to their size.

GPU Upload/Download

def Hesper.WebGPU.BufferOps.uploadFloatArray (device : Device) (data : Array Float) : IO Buffer

Upload Float32 array to GPU buffer

Creates a new buffer and uploads data.

@param device WebGPU device @param data Array of Float32 values @return GPU buffer containing the data

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.uploadTokens (device : Device) (tokens : Array Nat) : IO Buffer

Upload token IDs to GPU buffer

Creates a new buffer and uploads token data.

@param device WebGPU device @param tokens Array of token IDs @return GPU buffer containing the tokens

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.downloadFloatArray (device : Device) (buffer : Buffer) (numElements : Nat) : IO (Array Float)

Download Float32 array from GPU buffer

Reads data from GPU buffer back to CPU.

@param device WebGPU device @param buffer GPU buffer to read from @param numElements Number of Float32 elements to read @return Array of Float32 values

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.downloadLastLogits (device : Device) (logitsBuffer : Buffer) (batchSize seqLen vocabSize : Nat) : IO (Array Float)

Download logits and extract last position

Helper function for text generation: downloads logits from GPU and extracts the logits for the last token position.

@param device WebGPU device @param logitsBuffer GPU buffer containing logits [batch, seq_len, vocab_size] @param batchSize Batch size @param seqLen Sequence length @param vocabSize Vocabulary size @return Logits for last position [vocab_size]

Equations

• One or more equations did not get rendered due to their size.

Utilities

def Hesper.WebGPU.BufferOps.createLogitsBuffer (device : Device) (batchSize seqLen vocabSize : Nat) : IO Buffer

Create buffer for model outputs (logits)

@param device WebGPU device @param batchSize Batch size @param seqLen Sequence length @param vocabSize Vocabulary size @return Buffer sized for logits

Equations

• One or more equations did not get rendered due to their size.

def Hesper.WebGPU.BufferOps.printBufferInfo (buffer : Buffer) (name : String) : IO Unit

Print buffer info for debugging

@param buffer GPU buffer @param name Descriptive name

Equations

• Hesper.WebGPU.BufferOps.printBufferInfo buffer name = do Hesper.Logging.logVerbose (toString "[Buffer] " ++ toString name) pure ()