Threads and Atomics

A game engine wants every core it can get: physics on one, animation on another, asset decoding on the rest. WebAssembly has no instruction to start a thread. Instead, the host starts threads, Web Workers in a browser, and gives each one an instance of the same module, all sharing one memory. What WebAssembly provides is the rest: memories that can be shared, atomic instructions to access them safely, and instructions to wait and wake. This chapter covers them, and how a compiler and its runtime fit threads together.

#Shared memory23.1

A memory declared shared can be imported by several instances at once, each running on its own thread, all seeing the same bytes. A shared memory must have a maximum size, because engines reserve its whole address range up front, so it never moves when it grows:

(module
  (memory (export "memory") 1 100 shared))

On the JavaScript side, a shared memory's buffer is a SharedArrayBuffer instead of an ArrayBuffer, and it can be sent to a worker with postMessage, which shares it rather than copying it. The usual arrangement is for the host to create the memory, and pass it to every instance as an import:

const memory = new WebAssembly.Memory({ initial: 256, maximum: 16384, shared: true });
const module = await WebAssembly.compileStreaming(fetch('engine.wasm'));
for (let i = 0; i < navigator.hardwareConcurrency - 1; i++) {
  const worker = new Worker('worker.js');
  worker.postMessage({ module, memory, id: i });     // each worker instantiates the module
}

Each worker instantiates the same compiled module with the same memory, and calls an export to start its work.

#Atomic instructions23.2

Ordinary loads and stores to shared memory are allowed, but when two threads touch the same location and at least one writes, the result is a data race: each thread may see the other's writes late, partially, or in a surprising order. Atomic instructions access memory indivisibly and in a single order that every thread agrees on. They all live behind the fe prefix:

Every i32 form has an i64 counterpart. Atomic accesses must be naturally aligned, and an unaligned one traps, unlike every other memory access:

(module
  (memory 1 1 shared)
  (func (export "main") (result i32)
    (i32.atomic.load (i32.const 2))))

RuntimeError: operation does not support unaligned accesses

The atomic instructions also work on unshared memories, where they behave like ordinary accesses (with the alignment rule). That lets the same code run single-threaded without a separate build.

All atomic instructions are sequentially consistent: there is one global order of all atomic operations, consistent with each thread's program order. That is the strongest and simplest ordering, matching C++'s memory_order_seq_cst. A proposal for weaker, cheaper orderings is in progress.

#Threads counting together23.3

Here are four threads each adding 1 to a shared counter 100,000 times. The module is the same for every thread:

(module
  (import "env" "memory" (memory 1 1 shared))
  (func (export "count") (param $n i32)
    (loop $again
      (drop (i32.atomic.rmw.add (i32.const 0) (i32.const 1)))
      (local.set $n (i32.sub (local.get $n) (i32.const 1)))
      (br_if $again (local.get $n))))
  (func (export "total") (result i32)
    (i32.atomic.load (i32.const 0))))

The worker instantiates it with the memory it is given, counts, and reports back:

import { parentPort, workerData } from 'node:worker_threads';

const { module, memory } = workerData;
const instance = new WebAssembly.Instance(module, { env: { memory } });
instance.exports.count(100000);
parentPort.postMessage('done');

And the main thread creates the memory, compiles the module once, and starts the workers:

import fs from 'node:fs';
import { Worker } from 'node:worker_threads';

const memory = new WebAssembly.Memory({ initial: 1, maximum: 1, shared: true });
const module = new WebAssembly.Module(fs.readFileSync('counter.wasm'));

const workers = Array.from({ length: 4 },
  () => new Worker('./worker.mjs', { workerData: { module, memory } }));
await Promise.all(workers.map((w) => new Promise((done) => w.once('message', done))));
workers.forEach((w) => w.terminate());

const main = new WebAssembly.Instance(module, { env: { memory } });
console.log(main.exports.total());

400000

With an ordinary i32.load, i32.add and i32.store in place of the atomic add, the threads would overwrite each other's updates, and the total would come out lower, by a different amount on each run. (That version is not shown, since its output is not reproducible.)

Node's worker_threads stand in for Web Workers here. In a browser, the worker would receive the module and memory through postMessage rather than workerData, but everything else is the same.

#Waiting and waking23.4

Spinning on an atomic load wastes a core. memory.atomic.wait32 puts a thread to sleep instead. It takes an address, an expected value, and a timeout in nanoseconds (−1 for none). If the location does not hold the expected value, it returns 1 at once; otherwise it sleeps until another thread calls memory.atomic.notify on that address (returning 0) or the timeout passes (returning 2):

(module
  (import "env" "print_i32" (func $print (param i32)))
  (memory 1 1 shared)
  (func (export "main")
    ;; the location holds 0, not 5: returns 1, "not equal", without sleeping
    (call $print (memory.atomic.wait32 (i32.const 0) (i32.const 5) (i64.const -1)))
    ;; it does hold 0: sleeps for the 1 ms timeout, then returns 2, "timed out"
    (call $print (memory.atomic.wait32 (i32.const 0) (i32.const 0) (i64.const 1000000)))
    ;; wakes nobody, since nobody is waiting: returns the number woken
    (call $print (memory.atomic.notify (i32.const 0) (i32.const 1)))))

1
2
0

The check and the sleep happen atomically, so a thread cannot miss a notification that arrives between checking the value and going to sleep. These two instructions are exactly what is needed to build mutexes, condition variables, semaphores and job queues, the same way futexes are used on Linux.

A browser does not allow the main thread to block, so wait traps there. The main thread must use JavaScript's Atomics.waitAsync, or avoid waiting altogether, which suits a game anyway: the main thread runs the frame loop, and hands work to workers.

#What is per thread23.5

Each thread has its own instance, so anything that belongs to an instance rather than to the shared memory is automatically per thread:

• Globals are per instance, so a mutable global is thread-local storage. The shadow stack pointer (Chapter 10) is a global, so each thread automatically has its own. Each thread does need its own area of memory for its stack, which the runtime allocates when the thread starts and stores in that thread's stack pointer global.
• Tables are per instance, unless imported and shared. A function pointer that one thread stores in memory must mean the same function on every thread, so either every instance must put functions at the same table indices, which they do if the table is filled only by the module's own element segments, or tables must be shared too.
• Data segments are applied again by every instance, which would reset the shared memory each time a thread starts. Toolchains make data segments passive (Chapter 9), and have one thread copy them in with memory.init, guarded by an atomic flag.

#What you have seen23.6

Threads are host workers, each with an instance of the same module, sharing a shared memory that has a fixed maximum size. Atomic loads, stores and read-modify-write operations access it safely, in a single sequentially consistent order, and must be aligned. wait and notify let threads sleep instead of spinning, but not on a browser's main thread. Globals are per thread, so the stack pointer and thread-local storage come naturally, while data segments and tables need care. Pages must be cross-origin isolated to use any of it.

That ends Part VI. Part VII brings everything together: how to map a language onto WebAssembly, and how engines turn the result into fast machine code.