census.nu

packages/swarm/src/census.nu — lightweight membership gossip for the swarm.

The compute layer (dist/job over dist/ring) needs every node to agree on the live worker set: a worker must know it owns a key, and the coordinator must know which pubkeys to route to. The full SWIM table (net/membership.nu) is the heavy, churn-hardened answer; this is the small one a compute cluster actually needs — a HELLO announcement that feeds the consistent-hash ring.

• a node joins by broadcasting HELLO(role, want=1) to the relay group;
• everyone who hears it adds the node to their ring (workers only) and, if

want, replies with their own HELLO so the newcomer learns them too;

• the ring converges, and "join the cluster" is literally "run the binary".

Roles: a WORKER owns keys and executes handlers, so it joins everyone's ring; a CLIENT (the coordinator) only submits, so it is never added to the ring — otherwise it would own keys it has no handler for and silently drop results.

The codec and the membership set are pure; only the pump touches transport.

API

@ census_hello_t → i

@ role_client → i

@ role_worker → i

@ hello_build i id i role i want ( Vec u ) pubkey → ( Vec u )

HELLO wire: [3][id:8][role:1][want:1][pklen:2][pubkey…]

: Hello { i id i role i want ( Vec u ) pubkey }

@ hello_free Hello h → v

@ hello_decode ( Vec u ) buf → Hello

: Member { ( Vec u ) pubkey i id }

: Roster { ( Vec s ) members } // *Member

@ roster_new → *Roster

@ roster_free * Roster r → v

@ roster_has * Roster r ( Vec u ) pubkey → b

@ roster_count * Roster r → i

@ roster_add * Roster r * Ring ring ( Vec u ) pubkey i id i vnodes → b

Fold a worker into the roster + ring, once. Returns T if newly added.

---

work.nu

packages/swarm-mcp/src/work.nu — the distributed map-reduce the cluster runs.

A task is: evaluate an expression kernel (expr.nu) over an integer range [lo, hi) and fold the results with a reduce op. The coordinator shards the range; dist/ring routes each chunk to its owning worker; the worker parses the kernel once and folds it over its sub-range; the coordinator combines the partial folds. Every reduce op is associative, so sharding is exact.

reduce op : 0 sum · 1 product · 2 min · 3 max · 4 count (of truthy map) chunk : [op:1][lo:8 BE][hi:8 BE][expr bytes…] result : [value:8 BE]

API

@ red_sum → i

@ red_product → i

@ red_min → i

@ red_max → i

@ red_count → i

@ red_id i op → i

The identity (empty-range) value for a reduce op.

@ red_fold i op i acc i v → i

Fold one mapped value into the accumulator (per element).

@ red_combine i op i a i b → i

Combine two partial folds (across chunks). count combines by summing the per-chunk counts; every other op combines like its element fold.

@ chunk_payload i op i lo i hi ( Vec u ) expr → ( Vec u )

@ result_encode i value → ( Vec u )

@ result_decode ( Vec u ) p → i

@ kernel_handler → ( @ ( Vec u ) ( Vec u ) )

: Chunk { i lo i hi }

@ shard i lo i hi i n → ( Vec s )

@ shard_free ( Vec s ) chunks → v

@ chunk_key i idx → ( Vec u )

A ring key for chunk index i: distinct chunks hash to distinct ring points.

@ reduce_op_of s name i fallback → i

True for a recognised reduce-op name; sets *op. Keeps the MCP layer thin.

@ reduce_op_name i op → s

---

expr.nu

packages/swarm-mcp/src/expr.nu — the phase-1 "kernel": a small, regular integer-expression language the cluster evaluates per element.

A workload is a map-reduce: the coordinator ships an expression in one variable x plus a range and a reduce op; each worker parses the expression once, evaluates it for every x in its sub-range, and folds the results. The language is deliberately small and regular so a language model can write it without docs — and it is the natural precursor to phase 2, where the kernel is arbitrary NURL compiled to wasm instead of interpreted here.

expr := ternary ternary := logic ( '?' expr ':' ternary )? logic := compare ( ('&'|'|') compare ) compare := addsub ( ('<'|'<='|'>'|'>='|'=='|'!=') addsub )? addsub := muldiv ( ('+'|'-') muldiv ) muldiv := unary ( (''|'/'|'%') unary ) unary := '-' unary | primary primary := INT | 'x' | '(' expr ')' | ('min'|'max') '(' expr ',' expr ')' | 'abs' '(' expr ')'

All arithmetic is i64 (truncated division; division/mod by zero yields 0). Comparisons and '&' '|' yield 0/1; any non-zero is "true".

Tokens are kept in two parallel int vectors; the parser builds a flat stride-4 node arena (tag,a,b,c) and returns the root index — the resp.nu arena pattern, so nesting needs no per-node allocation.

API

@ expr_tokenize ( Vec u ) src ( Vec i ) tk ( Vec i ) tv → b

: EParser

: EParser {
    ( Vec i ) tk
    ( Vec i ) tv
    i pos
    ( Vec i ) arena  // stride-4: tag,a,b,c
    b ok
}

@ expr_parse ( Vec u ) src * EParser p → i

Parse src → (arena, root). On any error, ok=0; the caller checks it. The arena is returned via the EParser; the root index via the return value.

@ eparser_free * EParser p → v

Frees the parser's arena/token vectors AND the struct itself.

@ expr_eval * EParser p i node i x → i

---

main.nu

packages/swarm-mcp/src/main.nu — swarm-mcp: an MCP-controlled distributed compute engine. A language model sets a workload over MCP — an expression kernel in x plus a range and a reduce op — and the cluster evaluates it distributed; the model reads running tasks and finished results back.

swarm-mcp relay 0.0.0.0 47700 [--v] # the meeting point (one per cluster) swarm-mcp worker <host> <port> # join as a compute node swarm-mcp mcp <host> <port> # MCP server (stdio) → drives the cluster swarm-mcp submit <host> <port> <reduce> <lo> <hi> <expr> # manual CLI submit

The cluster layer (membership → ring → job dispatch) is the swarm package's; here every worker registers ONE generic kernel handler (work.nu) that interprets the submitted expression (expr.nu), so any normal-operation workload runs without recompiling a worker. Phase 2 swaps the interpreter for a NURL→wasm kernel; the protocol and MCP surface stay the same.

API

@ swarm_vnodes → i

@ kind_kernel → i

@ swarm_group_id → ( Vec u )

Fixed 32-byte multicast group (the relay's documented contract).

@ pk_from_id i id → ( Vec u )

A 32-byte opaque routing pubkey derived deterministically from a node id.

: Swarm

: Swarm {
    s transport  // *Transport
    s ring  // *Ring
    s roster  // *Roster
    s job  // *JobNode
    ( Vec u ) self_pk
    i self_id
    i role
    ( Vec u ) group
}

@ swarm_new RelayClient rc i id i role → *Swarm

@ swarm_free * Swarm sw → v

@ swarm_join_group * Swarm sw → v

@ swarm_announce * Swarm sw i want → v

@ swarm_on_hello * Swarm sw Hello h → v

@ swarm_pump * Swarm sw i max → v

@ swarm_discover * Swarm sw i rounds → v

@ run_relay s host i port i vflag → i

@ run_worker s host i port i id i rounds → i

@ cluster_submit * Swarm sw i op i lo i hi ( Vec u ) expr i nchunks → ( Vec i )

@ tids_ready * Swarm sw ( Vec i ) tids → b

All chunk results present? (cluster job results are recorded by task-id.)

@ tids_combine * Swarm sw i op ( Vec i ) tids → i

Combine all chunk results with the reduce op (assumes ready).

@ nchunks_for i nworkers → i

@ run_submit s host i port i op i lo i hi s expr → i

: Task

: Task {
    i id
    ( Vec u ) expr  // kernel bytes
    i lo
    i hi
    i op
    i nchunks
    ( Vec i ) tids
    i done
    i result
}

: McpState

: McpState {
    s swarm  // *Swarm coordinator
    ( Vec s ) tasks  // *Task
    i next_id
}

@ task_new i tid ( Vec u ) ex i a i b i o i nc ( Vec i ) ts → *Task

Constructor: param names deliberately differ from the field names — a store LHS . t lo whose value-name also matched the field would compile to an array-index store, not a field store.

: ~ i g_mcp 0

@ mcp_swarm → *Swarm

@ mcp_pump i rounds → v

@ task_refresh * Task t → v

Refresh one task's status; combine if every chunk has landed.

@ task_find i id → s

@ task_to_json * Task t → Json

Build the JSON-as-text result body for one task (LLM-readable + parseable).

@ tool_result_json Json o → Json

@ tool_submit Json args → Json

@ tool_list Json args → Json

@ tool_result Json args → Json

@ ms_prop Json props s name s ty s desc → v

@ ms_schema_submit → Json

@ ms_schema_result → Json

@ ms_schema_empty → Json

@ build_tools_list → ( Vec Json )

@ dispatch_tool s name Json args → Json

@ handle_initialize Json id → Json

@ handle_ping Json id → Json

@ handle_tools_list Json id → Json

@ handle_tools_call Json id Json params → Json

@ handle_unknown Json id s method → Json

@ dispatch Json req → ?Json

@ stdio_loop → v

@ run_mcp s host i port → i

@ usage → v

@ arg_int i idx → i

@ arg_eq i idx s lit → b

@ main → i