census.nu

packages/swarm/src/census.nu — lightweight membership gossip for the swarm.

The compute layer (dist/job over dist/ring) needs every node to agree on the live worker set: a worker must know it owns a key, and the coordinator must know which pubkeys to route to. The full SWIM table (net/membership.nu) is the heavy, churn-hardened answer; this is the small one a compute cluster actually needs — a HELLO announcement that feeds the consistent-hash ring.

• a node joins by broadcasting HELLO(role, want=1) to the relay group;
• everyone who hears it adds the node to their ring (workers only) and, if

want, replies with their own HELLO so the newcomer learns them too;

• the ring converges, and "join the cluster" is literally "run the binary".

Roles: a WORKER owns keys and executes handlers, so it joins everyone's ring; a CLIENT (the coordinator) only submits, so it is never added to the ring — otherwise it would own keys it has no handler for and silently drop results.

The codec and the membership set are pure; only the pump touches transport.

API

@ census_hello_t → i

@ role_client → i

@ role_worker → i

@ hello_build i id i role i want ( Vec u ) pubkey → ( Vec u )

HELLO wire: [3][id:8][role:1][want:1][pklen:2][pubkey…]

: Hello { i id i role i want ( Vec u ) pubkey }

@ hello_free Hello h → v

@ hello_decode ( Vec u ) buf → Hello

: Member { ( Vec u ) pubkey i id }

: Roster { ( Vec s ) members } // *Member

@ roster_new → *Roster

@ roster_free * Roster r → v

@ roster_has * Roster r ( Vec u ) pubkey → b

@ roster_count * Roster r → i

@ roster_add * Roster r * Ring ring ( Vec u ) pubkey i id i vnodes → b

Fold a worker into the roster + ring, once. Returns T if newly added.

---

work.nu

packages/swarm-mcp/src/work.nu — the distributed map-reduce the cluster runs.

A task is: evaluate an expression kernel (expr.nu) over an integer range [lo, hi) and fold the results with a reduce op. The coordinator shards the range; dist/ring routes each chunk to its owning worker; the worker parses the kernel once and folds it over its sub-range; the coordinator combines the partial folds. Every reduce op is associative, so sharding is exact.

reduce op : 0 sum · 1 product · 2 min · 3 max · 4 count (of truthy map) chunk : [op:1][lo:8 BE][hi:8 BE][expr bytes…] result : [value:8 BE]

API

@ red_sum → i

@ red_product → i

@ red_min → i

@ red_max → i

@ red_count → i

@ red_id i op → i

The identity (empty-range) value for a reduce op.

@ red_fold i op i acc i v → i

Fold one mapped value into the accumulator (per element).

@ red_combine i op i a i b → i

Combine two partial folds (across chunks). count combines by summing the per-chunk counts; every other op combines like its element fold.

@ chunk_payload i op i lo i hi ( Vec u ) expr → ( Vec u )

@ result_encode i value → ( Vec u )

@ result_decode ( Vec u ) p → i

@ kernel_handler → ( @ ( Vec u ) ( Vec u ) )

: Chunk { i lo i hi }

@ shard i lo i hi i n → ( Vec s )

@ shard_free ( Vec s ) chunks → v

@ chunk_key i idx → ( Vec u )

A ring key for chunk index i: distinct chunks hash to distinct ring points.

@ reduce_op_of s name i fallback → i

True for a recognised reduce-op name; sets *op. Keeps the MCP layer thin.

@ reduce_op_name i op → s

---

expr.nu

packages/swarm-mcp/src/expr.nu — the phase-1 "kernel": a small, regular integer-expression language the cluster evaluates per element.

A workload is a map-reduce: the coordinator ships an expression in one variable x plus a range and a reduce op; each worker parses the expression once, evaluates it for every x in its sub-range, and folds the results. The language is deliberately small and regular so a language model can write it without docs — and it is the natural precursor to phase 2, where the kernel is arbitrary NURL compiled to wasm instead of interpreted here.

expr := ternary ternary := logic ( '?' expr ':' ternary )? logic := compare ( ('&'|'|') compare ) compare := addsub ( ('<'|'<='|'>'|'>='|'=='|'!=') addsub )? addsub := muldiv ( ('+'|'-') muldiv ) muldiv := unary ( (''|'/'|'%') unary ) unary := '-' unary | primary primary := INT | 'x' | '(' expr ')' | ('min'|'max') '(' expr ',' expr ')' | 'abs' '(' expr ')'

All arithmetic is i64 (truncated division; division/mod by zero yields 0). Comparisons and '&' '|' yield 0/1; any non-zero is "true".

Tokens are kept in two parallel int vectors; the parser builds a flat stride-4 node arena (tag,a,b,c) and returns the root index — the resp.nu arena pattern, so nesting needs no per-node allocation.

API

@ expr_tokenize ( Vec u ) src ( Vec i ) tk ( Vec i ) tv → b

: EParser

: EParser {
    ( Vec i ) tk
    ( Vec i ) tv
    i pos
    ( Vec i ) arena  // stride-4: tag,a,b,c
    b ok
}

@ expr_parse ( Vec u ) src * EParser p → i

Parse src → (arena, root). On any error, ok=0; the caller checks it. The arena is returned via the EParser; the root index via the return value.

@ eparser_free * EParser p → v

Frees the parser's arena/token vectors AND the struct itself.

@ expr_eval * EParser p i node i x → i

---

buildwasm.nu

packages/swarm-mcp/src/buildwasm.nu — compile NURL source to a wasm module via the NURL build API, so the MCP server itself can accept a kernel as source (compute_submit_kernel) instead of requiring the caller to pre-compile.

POST {source, filename} to <NURL_BUILD_API>/build_wasm. A direct nurlapi can answer with the raw wasm bytes; the public playground proxy answers with JSON carrying wasm_base64 (and nurlc_errors on failure). Both are handled.

$NURL_BUILD_API build service base URL (default https://play.nurl-lang.org)

API

@ build_api_url → String

@ wrap_kernel s source i op → String

Wrap a bare kernel into a complete wasm-ready program for reduce op op.

@ compile_to_wasm s source → !( Vec u ) String

Compile NURL source to a wasm module. Ok = module bytes; Err = a human-readable transport/compile error (suitable to hand back to the model).

---

main.nu

packages/swarm-mcp/src/main.nu — swarm-mcp: an MCP-controlled distributed compute engine. A language model sets a workload over MCP — an expression kernel in x plus a range and a reduce op — and the cluster evaluates it distributed; the model reads running tasks and finished results back.

swarm-mcp relay 0.0.0.0 47700 [--v] # the meeting point (one per cluster) swarm-mcp worker <host> <port> # join as a compute node swarm-mcp mcp <host> <port> # MCP server (stdio) → drives the cluster swarm-mcp submit <host> <port> <reduce> <lo> <hi> <expr> # manual CLI submit

The cluster layer (membership → ring → job dispatch) is the swarm package's; here every worker registers ONE generic kernel handler (work.nu) that interprets the submitted expression (expr.nu), so any normal-operation workload runs without recompiling a worker. Phase 2 swaps the interpreter for a NURL→wasm kernel; the protocol and MCP surface stay the same.

API

@ swarm_vnodes → i

@ kind_kernel → i

@ swarm_group_id → ( Vec u )

Fixed 32-byte multicast group (the relay's documented contract).

@ pk_from_id i id → ( Vec u )

A 32-byte opaque routing pubkey derived deterministically from a node id.

: Swarm

: Swarm {
    s transport  // *Transport
    s ring  // *Ring
    s roster  // *Roster
    s job  // *JobNode
    ( Vec u ) self_pk
    i self_id
    i role
    ( Vec u ) group
}

@ swarm_new RelayClient rc i id i role → *Swarm

@ swarm_free * Swarm sw → v

@ swarm_join_group * Swarm sw → v

@ swarm_announce * Swarm sw i want → v

@ swarm_on_hello * Swarm sw Hello h → v

@ swarm_pump * Swarm sw i max → v

@ swarm_discover * Swarm sw i rounds → v

@ run_relay s host i port i vflag → i

@ run_worker s host i port i id i rounds → i

@ cluster_submit * Swarm sw i op i lo i hi ( Vec u ) expr i nchunks → ( Vec i )

@ cluster_submit_wasm * Swarm sw i lo i hi ( Vec u ) wasm i nchunks → ( Vec i )

Shard a wasm-kernel task: ship the compiled module + each sub-range to its ring owner under kind_wasm. The module bytes ride every chunk (workers cache by content hash, so it is written once per worker).

@ tids_ready * Swarm sw ( Vec i ) tids → b

All chunk results present? (cluster job results are recorded by task-id.)

@ tids_combine * Swarm sw i op ( Vec i ) tids → i

Combine all chunk results with the reduce op (assumes ready).

@ nchunks_for i nworkers → i

@ run_submit s host i port i op i lo i hi s expr → i

@ nchunks_wasm i nworkers → i

Chunk count for a wasm task: fewer chunks than the expression path (each chunk ships the module and spawns a wasmtime process), but enough to spread across the ring.

@ run_runwasm s host i port i op i lo i hi s wasmpath → i

: Task

: Task {
    i id
    ( Vec u ) expr  // kernel bytes
    i lo
    i hi
    i op
    i nchunks
    ( Vec i ) tids
    i done
    i result
}

: McpState

: McpState {
    s swarm  // *Swarm coordinator
    ( Vec s ) tasks  // *Task
    i next_id
}

@ task_new i id ( Vec u ) expr i lo i hi i op i nchunks ( Vec i ) tids → *Task

Constructor. The params share the field names (lo, hi, …); = . t lo lo is a field store — the field-store/local-shadow miscompile this used to work around was fixed in the compiler (NURL v0.10.4).

: ~ i g_mcp 0

@ mcp_swarm → *Swarm

@ mcp_pump i rounds → v

@ task_refresh * Task t → v

Refresh one task's status; combine if every chunk has landed.

@ task_find i id → s

@ task_to_json * Task t → Json

Build the JSON-as-text result body for one task (LLM-readable + parseable).

@ tool_result_json Json o → Json

@ tool_submit Json args → Json

@ tool_run_wasm Json args → Json

@ tool_submit_kernel Json args → Json

compute_submit_kernel: the server compiles NURL source → wasm itself (via the build API), then runs it — the model hands over a kernel as plain NURL.

@ tool_list Json args → Json

@ tool_result Json args → Json

@ ms_prop Json props s name s ty s desc → v

@ ms_schema_submit → Json

@ ms_schema_result → Json

@ ms_schema_empty → Json

@ ms_schema_run_wasm → Json

@ ms_schema_submit_kernel → Json

@ build_tools_list → ( Vec Json )

@ dispatch_tool s name Json args → Json

@ handle_initialize Json id → Json

@ handle_ping Json id → Json

@ handle_tools_list Json id → Json

@ handle_tools_call Json id Json params → Json

@ handle_unknown Json id s method → Json

@ dispatch Json req → ?Json

@ stdio_loop → v

@ run_mcp s host i port → i

@ usage → v

@ arg_int i idx → i

@ arg_eq i idx s lit → b

@ main → i

---

wasmkernel.nu

packages/swarm-mcp/src/wasmkernel.nu — phase 2: run a NURL-compiled wasm kernel over a chunk.

The language model writes a kernel as ordinary NURL — anything, not just an expression — and compiles it (via the NURL build service) to a wasm32-wasi module whose main reads lo hi from argv, folds the kernel over [lo, hi), and prints the partial. The coordinator ships that module plus a sub-range to each worker; the worker runs it under wasmtime and returns the partial, which the coordinator combines with the reduce op exactly as in phase 1.

chunk : [lo:8 BE][hi:8 BE][wasm module bytes…] (the reduce op is baked into the module by the wrapper, so the partial is final per chunk)

The module is cached on each worker by a content hash, so re-running the same kernel (every chunk of a task, and across tasks) writes the .wasm only once.

API

@ kind_wasm → i

@ wasm_chunk_payload i lo i hi ( Vec u ) wasm → ( Vec u )

@ wasm_handler → ( @ ( Vec u ) ( Vec u ) )

The worker handler for a wasm chunk: cache the module by hash, run it.