quant.nu

packages/gguf/src/quant.nu — host-side scalar quantisation encoders.

The write-side mirror of dequant.nu: turn the canonical f32-LE buffer (the exact form gguf_dequant produces) into GGUF tensor payload bytes, following ggml's reference quantisers bit for bit. dequant.nu is the decode oracle these encoders are tested against — encode → decode must land within the format's own rounding.

( gq_f16_encode f32le ) → !( Vec u ) String f32-LE → f16-LE (RNE) ( gq_bf16_encode f32le ) → !( Vec u ) String f32-LE → bf16-LE (RNE) ( gq_q8_0_encode f32le ) → !( Vec u ) String f32-LE → Q8_0 blocks

Layouts (ggml, exactly): Q8_0: 34-byte block = f16 d + 32 signed bytes, d = amax/127, q = roundf(x/d) — quantize_row_q8_0_ref.

---

gguf.nu

packages/gguf/src/gguf.nu — a bulletproof GGUF container parser.

GGUF is the ggml-ecosystem tensor container (llama.cpp, whisper.cpp, stable-diffusion.cpp): a little-endian header, typed metadata key/values, a tensor table, then an aligned data section. This module parses versions 2 and 3, mmap-backed, without ever loading tensor data into memory — tensor bytes are addressed lazily straight out of the mapping.

Trust model: the file is UNTRUSTED input. Every count, length and offset is validated against the actual file size BEFORE any allocation or read depends on it; allocations are proportional to bytes actually consumed from the file, never to what a header merely claims. A malformed or truncated file is an error, never a crash, a hang, or an unbounded allocation.

( gguf_open path ) → !Gguf String mmap + parse ( gguf_parse_bytes data ) → !Gguf String parse a buffer (the Gguf keeps data) ( gguf_close g ) → v early release (optional) ( gguf_kvs g ) ( gguf_tensors g ) → the metadata / tensor tables (borrowed) ( gguf_none ) / ( gguf_is_open g ) → an empty slot, and the test for one ( gguf_n_kv g ) ( gguf_n_tensors g ) → i ( gguf_find_kv g key ) → i index, -1 when absent ( gguf_kv_int_or g key def ) → i any int/bool-typed KV ( gguf_kv_f_or g key def ) → f f32/f64-typed KV ( gguf_kv_str_or g key def ) → s BORROWED from g ( gguf_find_tensor g name ) → i index, -1 when absent ( gguf_tensor_ptr g t ) → *u BORROWED tensor bytes ( gguf_type_name t ) ( gguf_type_blck t ) ( gguf_type_size t )

Layout invariants enforced at parse time: magic, version ∈ {2,3}, sane KV/tensor counts, in-bounds strings, no nested arrays, unique KV keys and tensor names, power-of-two alignment, dims ≥ 1 with overflow-checked element counts, aligned tensor offsets, and every sizable tensor fully inside the data section.

A Gguf is a handle: every copy is the same open file, and the last owner unmaps it (or lets go of the buffer it was parsed from). Nothing here is released by hand.

API

: i GGUF_VT_U8 0

── GGUF metadata value types (spec) ────────────────────────────────

: i GGUF_VT_I8 1

: i GGUF_VT_U16 2

: i GGUF_VT_I16 3

: i GGUF_VT_U32 4

: i GGUF_VT_I32 5

: i GGUF_VT_F32 6

: i GGUF_VT_BOOL 7

: i GGUF_VT_STR 8

: i GGUF_VT_ARR 9

: i GGUF_VT_U64 10

: i GGUF_VT_I64 11

: i GGUF_VT_F64 12

@ gguf_type_name i t → s

── ggml tensor types ─────────────────────────────────────────────── Name for a ggml type id; ? for ids this parser has no name for.

@ gguf_type_blck i t → i

Elements per quantisation block; 0 when this parser cannot size the type (unknown or not-yet-tabled id — such tensors are still listed, but their byte extent cannot be verified or dequantised).

@ gguf_type_size i t → i

Bytes per quantisation block; 0 when unknown (pairs with gguf_type_blck).

: GgufKv

: GgufKv {
    String key
    i vt
    i ival
    f fval
    String sval
    i atype
    ( Vec i ) ai
    ( Vec f ) af
    ( Vec String ) astr
}

One metadata key/value. vt discriminates which payload field holds the value: int family (u8..u64/i64/bool) → ival, f32/f64 → fval, string → sval, array → atype + one of ai/af/astr.

: GgufTensor

: GgufTensor {
    String name
    i gtype
    i nd
    i d0
    i d1
    i d2
    i d3
    i nelems
    i offset
    i nbytes
}

One tensor-table entry. offset is RELATIVE to the data section (exactly as stored in the file); absolute bytes are reached through gguf_tensor_ptr. nbytes is -1 when the type cannot be sized.

: GgufImpl

: GgufImpl {
    i version
    i align
    ( Vec GgufKv ) kvs
    ( Vec GgufTensor ) tensors
    i data_off
    i data_size
    * u map
    i map_size
    b from_mmap
    ( Vec u ) buf  // the bytes `map` points into, when they are not a mapping
}

% Drop GgufImpl

% Drop GgufImpl {
    unsafe @ drop GgufImpl g → v {
        ? . g from_mmap { : i32 _u ( munmap . g map . g map_size ) } {}
    }
}

The mapping is the raw resource: its last owner unmaps it, as gguf_close did. The tables and the buffer go with the drop glue.

: Gguf { s ctl }

A Gguf is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same open file, and the last owner releases it.

@ Gguf_drop sink Gguf h → v

@ gguf_none → Gguf

A Gguf that holds no file — for a slot that may be empty (a model kept open across a streamed merge, or none). gguf_is_open tells them apart.

@ gguf_is_open Gguf g → b

: GCur

: GCur {
    * u p
    i n
    i off
    b fail
}

── bounded little-endian cursor over the raw mapping ─────────────── Every read checks the remaining byte budget first; the first out-of-bounds read latches fail and all subsequent reads return zero values, so a parse can run to a checkpoint and test fail once.

@ gguf_close sink Gguf g → v

Let go of g now rather than at the end of its owner's scope; the last owner unmaps the file.

---

write.nu

packages/gguf/src/write.nu — a GGUF v3 writer/builder.

The reader's round-trip partner: build typed metadata and tensors in memory, then serialise a spec-exact GGUF v3 image. Powers the test suite (write → parse → compare, bit for bit) and gives the ecosystem an export path (tensor snapshots, converted models, fixtures).

( gw_new align ) → !GgufW String ( gw_kv_u32 w key v ) ( gw_kv_i32 … ) ( gw_kv_u64 … ) ( gw_kv_f32 w key x ) ( gw_kv_f64 … ) ( gw_kv_bool … ) ( gw_kv_str w key val ) ( gw_kv_arr_i32 w key vals ) — ( Vec i ), borrowed ( gw_kv_arr_f32 w key vals ) — ( Vec f ), borrowed ( gw_kv_arr_str w key vals ) — ( Vec String ), borrowed ( gw_tensor w name gt nd d0 d1 d2 d3 bytes ) → !v String ( gw_finish w ) → ( Vec u ) the file image ( gw_write w path ) → !v String ( gw_free w ) — early release (optional)

gw_new emits general.alignment itself (so the image is self-describing) — callers must not add that key again. gw_tensor validates the payload length against the declared type/dims and pads the data section to the alignment, exactly as the parser will demand on the way back in.

── The STREAMING writer ──────────────────────────────────────────── gw_ builds the whole image in memory — fine for fixtures, wrong for a multi-gigabyte model conversion on a machine whose RAM the model exceeds. The gws_ twin writes straight to a file in three phases: declare every KV and tensor (name/type/shape only), then gws_begin_data serialises the header + metadata + tensor table (all offsets are computable from the declarations alone), then payloads stream in DECLARATION ORDER through gws_data — any chunking, even byte at a time — and gws_finish verifies nothing is missing. Memory stays at the metadata + one caller chunk, whatever the file size.

( gws_create path align ) → !GgufS String ( gws_kv_u32 s key v ) … same KV family as gw_kv_* ( gws_tensor s name gt nd d0 d1 d2 d3 ) → !v String ( gws_begin_data s ) → !v String ( gws_data s bytes ) → !v String next payload bytes ( gws_finish s ) → !v String all payloads complete? ( gws_free s ) early release (optional)

GgufW and GgufS are handles: every copy is the same writer, and the last owner releases it — a streaming writer's file is closed then if gws_finish did not close it (any phase). Nothing here is released by hand.

API

: GgufWImpl

: GgufWImpl {
    i align
    i n_kv
    ( Vec u ) kvb
    ( Vec String ) tnames
    ( Vec i ) ttype
    ( Vec i ) tnd
    ( Vec i ) td0
    ( Vec i ) td1
    ( Vec i ) td2
    ( Vec i ) td3
    ( Vec i ) toff
    ( Vec u ) data
}

: GgufW { s ctl }

A GgufW is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same writer, and the last owner releases it.

@ GgufW_drop sink GgufW h → v

@ gw_write GgufW w__h s path → !v String

@ gw_free sink GgufW w → v

Let go of w now rather than at the end of its owner's scope.

: GgufSImpl

: GgufSImpl {
    i align
    i n_kv
    ( Vec u ) kvb
    ( Vec String ) tnames
    ( Vec i ) ttype
    ( Vec i ) tnd
    ( Vec i ) td0
    ( Vec i ) td1
    ( Vec i ) td2
    ( Vec i ) td3
    ( Vec i ) toff
    ( Vec i ) tbytes
    File fh  // the open output (its last owner closes it)
    // 0 = declaring KVs and tensors, 1 = streaming payloads, 2 = finished
    i phase
    i cur
    i cur_got
    i data_pos
    i data_size
}

: GgufS { s ctl }

A GgufS is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same writer, and the last owner releases it.

@ GgufS_drop sink GgufS h → v

@ gws_free sink GgufS s → v

Let go of s now rather than at the end of its owner's scope.

---

main.nu

gguf — inspect, verify and export GGUF model containers.

gguf info <file> one-line summary gguf dump <file> header + all metadata + tensors gguf kv <file> <key> print one metadata value gguf tensors <file> tensor table gguf verify <file> deep structural check (+ overlap) gguf export <file> <tensor> -o out.f32 dequantise to f32-LE gguf gen <out.gguf> write the reference sample file gguf selftest write → parse → compare, bit-exact

The parser treats every input as hostile: malformed, truncated or adversarial files produce an error message, never a crash, hang or unbounded allocation (see tests/gguf_test.sh, which proves this with a corruption battery).

API

: i GGUF_CLI_VERSION_MAJOR 0

: STCnt

: STCnt {
    i pass
    i fail
}

@ string_eq_raw String a s raw → b

String vs raw-literal equality (helper for selftest)

@ main → i

---

dequant.nu

packages/gguf/src/dequant.nu — host-side scalar dequantisation.

Reference implementation of the ggml block-quant decodings, pure NURL, no GPU dependency: this is the CPU decode path AND the golden oracle a GPU dequant kernel is verified against. Output is the upload-ready canonical form — a ( Vec u ) of little-endian IEEE-754 f32 values, one per tensor element, in storage order.

( gg_f16_to_f32_bits h ) → i IEEE half → f32 bit pattern ( gg_f16_to_f ) → f IEEE half → double ( gguf_dequant g idx ) → !( Vec u ) String f32-LE buffer ( gguf_dequant_f64 g idx ) → !( Vec f ) String convenience

Supported source types: F32, F64, F16, BF16, Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, and the K-quants Q4_K / Q5_K / Q6_K (the formats modern llama.cpp models actually ship in — a Q4_K_M file mixes Q4_K and Q6_K tensors). Anything else is a clean error naming the type — never a wrong answer. Layouts follow ggml exactly: Q4_0: 18-byte block = f16 scale d + 16 nibble bytes; elements 0..15 are the LOW nibbles, 16..31 the HIGH nibbles; value = d (q - 8). Q4_1: 20-byte block = f16 d + f16 min m + 16 nibble bytes; value = d q + m (same nibble order). Q8_0: 34-byte block = f16 d + 32 signed bytes; value = d * q.

API

@ gg_f16_to_f32_bits i h → i

IEEE-754 binary16 → binary32 — the stdlib bit-transport primitive (stdlib/std/floatbits.nu), kept under the historical gg_ name for the package's public dequant API.

@ gg_f16_to_f i h → f

@ gguf_dequant_range Gguf g i idx i first i count → !( Vec u ) String

Dequantise a contiguous ELEMENT RANGE of tensor #idx — [first, first+count) — to little-endian f32. first and count must be multiples of the type's block size (they always are for whole rows: ggml guarantees dim0 is a block multiple). This is what lets a consumer touch one row of a multi-gigabyte embedding table without expanding the whole tensor: nurllama dequantises exactly the token's row per step.

@ gguf_dequant Gguf g i idx → !( Vec u ) String

Dequantise the whole tensor.

@ gguf_dequant_f64 Gguf g i idx → !( Vec f ) String

Convenience: dequantise to host doubles (tests, small tensors).