quant.nu

packages/gguf/src/quant.nu — host-side scalar quantisation encoders.

The write-side mirror of dequant.nu: turn the canonical f32-LE buffer (the exact form gguf_dequant produces) into GGUF tensor payload bytes, following ggml's reference quantisers bit for bit. dequant.nu is the decode oracle these encoders are tested against — encode → decode must land within the format's own rounding.

( gq_f16_encode f32le ) → !( Vec u ) String f32-LE → f16-LE (RNE) ( gq_bf16_encode f32le ) → !( Vec u ) String f32-LE → bf16-LE (RNE) ( gq_q8_0_encode f32le ) → !( Vec u ) String f32-LE → Q8_0 blocks

Layouts (ggml, exactly): Q8_0: 34-byte block = f16 d + 32 signed bytes, d = amax/127, q = roundf(x/d) — quantize_row_q8_0_ref.

API

@ gq_f16_encode ( Vec u ) f32le → !( Vec u ) String

f32-LE → f16-LE, round-to-nearest-even per element.

@ gq_bf16_encode ( Vec u ) f32le → !( Vec u ) String

f32-LE → bf16-LE, round-to-nearest-even per element.

@ gq_q8_0_encode ( Vec u ) f32le → !( Vec u ) String

f32-LE → Q8_0 blocks: per 32 elements, d = amax/127 stored as f16, then 32 bytes of roundf(x * (127/amax)) two's-complement int8. ggml's quantize_row_q8_0_ref, including the id = d ? 1/d : 0 guard (an all-zero block stores d = 0 and 32 zero quants).

---

gguf.nu

packages/gguf/src/gguf.nu — a bulletproof GGUF container parser.

GGUF is the ggml-ecosystem tensor container (llama.cpp, whisper.cpp, stable-diffusion.cpp): a little-endian header, typed metadata key/values, a tensor table, then an aligned data section. This module parses versions 2 and 3, mmap-backed, without ever loading tensor data into memory — tensor bytes are addressed lazily straight out of the mapping.

Trust model: the file is UNTRUSTED input. Every count, length and offset is validated against the actual file size BEFORE any allocation or read depends on it; allocations are proportional to bytes actually consumed from the file, never to what a header merely claims. A malformed or truncated file is an error, never a crash, a hang, or an unbounded allocation.

( gguf_open path ) → !Gguf String mmap + parse ( gguf_parse_bytes data ) → !Gguf String parse a buffer (BORROWS data — keep it alive until gguf_close) ( gguf_close g ) → v unmap + free ( gguf_n_kv g ) ( gguf_n_tensors g ) → i ( gguf_find_kv g key ) → i index, -1 when absent ( gguf_kv_int_or g key def ) → i any int/bool-typed KV ( gguf_kv_f_or g key def ) → f f32/f64-typed KV ( gguf_kv_str_or g key def ) → s BORROWED from g ( gguf_find_tensor g name ) → i index, -1 when absent ( gguf_tensor_ptr g t ) → *u BORROWED tensor bytes ( gguf_type_name t ) ( gguf_type_blck t ) ( gguf_type_size t )

Layout invariants enforced at parse time: magic, version ∈ {2,3}, sane KV/tensor counts, in-bounds strings, no nested arrays, unique KV keys and tensor names, power-of-two alignment, dims ≥ 1 with overflow-checked element counts, aligned tensor offsets, and every sizable tensor fully inside the data section.

API

: i GGUF_VT_U8 0

── GGUF metadata value types (spec) ────────────────────────────────

: i GGUF_VT_I8 1

: i GGUF_VT_U16 2

: i GGUF_VT_I16 3

: i GGUF_VT_U32 4

: i GGUF_VT_I32 5

: i GGUF_VT_F32 6

: i GGUF_VT_BOOL 7

: i GGUF_VT_STR 8

: i GGUF_VT_ARR 9

: i GGUF_VT_U64 10

: i GGUF_VT_I64 11

: i GGUF_VT_F64 12

@ gguf_type_name i t → s

── ggml tensor types ─────────────────────────────────────────────── Name for a ggml type id; ? for ids this parser has no name for.

@ gguf_type_blck i t → i

Elements per quantisation block; 0 when this parser cannot size the type (unknown or not-yet-tabled id — such tensors are still listed, but their byte extent cannot be verified or dequantised).

@ gguf_type_size i t → i

Bytes per quantisation block; 0 when unknown (pairs with gguf_type_blck).

: GgufKv

: GgufKv {
    String key
    i vt
    i ival
    f fval
    String sval
    i atype
    ( Vec i ) ai
    ( Vec f ) af
    ( Vec String ) astr
}

One metadata key/value. vt discriminates which payload field holds the value: int family (u8..u64/i64/bool) → ival, f32/f64 → fval, string → sval, array → atype + one of ai/af/astr.

: GgufTensor

: GgufTensor {
    String name
    i gtype
    i nd
    i d0
    i d1
    i d2
    i d3
    i nelems
    i offset
    i nbytes
}

One tensor-table entry. offset is RELATIVE to the data section (exactly as stored in the file); absolute bytes are reached through gguf_tensor_ptr. nbytes is -1 when the type cannot be sized.

: Gguf

: Gguf {
    i version
    i align
    ( Vec GgufKv ) kvs
    ( Vec GgufTensor ) tensors
    i data_off
    i data_size
    * u map
    i map_size
    b from_mmap
    ( Vec u ) buf
}

: GCur

: GCur {
    * u p
    i n
    i off
    b fail
}

── bounded little-endian cursor over the raw mapping ─────────────── Every read checks the remaining byte budget first; the first out-of-bounds read latches fail and all subsequent reads return zero values, so a parse can run to a checkpoint and test fail once.

@ gguf_open s path → !*Gguf String

mmap-backed open (POSIX). Platforms without MAP_PRIVATE (wasm, win32) fall back to reading the whole file into an owned buffer — correct everywhere, mmap-lazy where it matters.

@ gguf_parse_bytes ( Vec u ) data → !*Gguf String

Parse an in-memory GGUF image. BORROWS data — the caller keeps the vec alive until gguf_close and frees it afterwards.

@ gguf_close * Gguf g → v

@ gguf_version * Gguf g → i

@ gguf_align * Gguf g → i

@ gguf_n_kv * Gguf g → i

@ gguf_n_tensors * Gguf g → i

@ gguf_data_size * Gguf g → i

@ gguf_find_kv * Gguf g s key → i

@ gguf_kv_int_or * Gguf g s key i def → i

@ gguf_kv_f_or * Gguf g s key f def → f

@ gguf_kv_str_or * Gguf g s key s def → s

BORROWED: the returned s points into g (or is def); valid until gguf_close. Do not free.

@ gguf_find_tensor * Gguf g s name → i

@ gguf_tensor_ptr * Gguf g GgufTensor t → *u

BORROWED pointer to a tensor's first byte inside the mapping; valid until gguf_close. Length is t.nbytes (when ≥ 0).

---

write.nu

packages/gguf/src/write.nu — a GGUF v3 writer/builder.

The reader's round-trip partner: build typed metadata and tensors in memory, then serialise a spec-exact GGUF v3 image. Powers the test suite (write → parse → compare, bit for bit) and gives the ecosystem an export path (tensor snapshots, converted models, fixtures).

( gw_new align ) → !*GgufW String ( gw_kv_u32 w key v ) ( gw_kv_i32 … ) ( gw_kv_u64 … ) ( gw_kv_f32 w key x ) ( gw_kv_f64 … ) ( gw_kv_bool … ) ( gw_kv_str w key val ) ( gw_kv_arr_i32 w key vals ) — ( Vec i ), borrowed ( gw_kv_arr_f32 w key vals ) — ( Vec f ), borrowed ( gw_kv_arr_str w key vals ) — ( Vec String ), borrowed ( gw_tensor w name gt nd d0 d1 d2 d3 bytes ) → !v String ( gw_finish w ) → ( Vec u ) the file image ( gw_write w path ) → !v String ( gw_free w )

gw_new emits general.alignment itself (so the image is self-describing) — callers must not add that key again. gw_tensor validates the payload length against the declared type/dims and pads the data section to the alignment, exactly as the parser will demand on the way back in.

── The STREAMING writer ──────────────────────────────────────────── gw_ builds the whole image in memory — fine for fixtures, wrong for a multi-gigabyte model conversion on a machine whose RAM the model exceeds. The gws_ twin writes straight to a file in three phases: declare every KV and tensor (name/type/shape only), then gws_begin_data serialises the header + metadata + tensor table (all offsets are computable from the declarations alone), then payloads stream in DECLARATION ORDER through gws_data — any chunking, even byte at a time — and gws_finish verifies nothing is missing. Memory stays at the metadata + one caller chunk, whatever the file size.

( gws_create path align ) → !GgufS String ( gws_kv_u32 s key v ) … same KV family as gw_kv_ ( gws_tensor s name gt nd d0 d1 d2 d3 ) → !v String ( gws_begin_data s ) → !v String ( gws_data s bytes ) → !v String next payload bytes ( gws_finish s ) → !v String all payloads complete? ( gws_free s ) close + free (any phase)

API

: GgufW

: GgufW {
    i align
    i n_kv
    ( Vec u ) kvb
    ( Vec String ) tnames
    ( Vec i ) ttype
    ( Vec i ) tnd
    ( Vec i ) td0
    ( Vec i ) td1
    ( Vec i ) td2
    ( Vec i ) td3
    ( Vec i ) toff
    ( Vec u ) data
}

@ gw_new i align → !*GgufW String

@ gw_kv_u32 * GgufW w s key i v → v

@ gw_kv_i32 * GgufW w s key i v → v

@ gw_kv_u64 * GgufW w s key i v → v

@ gw_kv_i64 * GgufW w s key i v → v

@ gw_kv_f32 * GgufW w s key f x → v

@ gw_kv_f64 * GgufW w s key f x → v

@ gw_kv_bool * GgufW w s key b v → v

@ gw_kv_str * GgufW w s key s val → v

@ gw_kv_arr_i32 * GgufW w s key ( Vec i ) vals → v

@ gw_kv_arr_f32 * GgufW w s key ( Vec f ) vals → v

@ gw_kv_arr_str * GgufW w s key ( Vec String ) vals → v

@ gw_tensor * GgufW w s name i gt i nd i d0 i d1 i d2 i d3 ( Vec u ) bytes → !v String

Add a tensor: validates the declared shape against the payload size with the same rules the parser enforces, aligns the data section, and records the entry. Unused trailing dims pass 1.

@ gw_finish * GgufW w → ( Vec u )

Serialise the complete GGUF v3 image.

@ gw_write * GgufW w s path → !v String

@ gw_free * GgufW w → v

: GgufS

: GgufS {
    i align
    i n_kv
    ( Vec u ) kvb
    ( Vec String ) tnames
    ( Vec i ) ttype
    ( Vec i ) tnd
    ( Vec i ) td0
    ( Vec i ) td1
    ( Vec i ) td2
    ( Vec i ) td3
    ( Vec i ) toff
    ( Vec i ) tbytes
    s fh
    // 0 = declaring KVs and tensors, 1 = streaming payloads, 2 = finished
    i phase
    i cur
    i cur_got
    i data_pos
    i data_size
}

@ gws_create s path i align → !*GgufS String

@ gws_kv_u32 * GgufS s s key i v → v

@ gws_kv_i32 * GgufS s s key i v → v

@ gws_kv_u64 * GgufS s s key i v → v

@ gws_kv_i64 * GgufS s s key i v → v

@ gws_kv_f32 * GgufS s s key f x → v

@ gws_kv_f64 * GgufS s s key f x → v

@ gws_kv_bool * GgufS s s key b v → v

@ gws_kv_str * GgufS s s key s val → v

@ gws_kv_arr_i32 * GgufS s s key ( Vec i ) vals → v

@ gws_kv_arr_f32 * GgufS s s key ( Vec f ) vals → v

@ gws_kv_arr_str * GgufS s s key ( Vec String ) vals → v

@ gws_tensor * GgufS s s name i gt i nd i d0 i d1 i d2 i d3 → !v String

Declare a tensor: shape only — the payload streams in later, in declaration order. The data-section offset is fixed here, which is what lets the whole tensor table serialise before any payload byte exists.

@ gws_begin_data * GgufS s → !v String

Serialise the header, every KV and the whole tensor table, pad to the alignment, and switch to the payload phase.

@ gws_data * GgufS s ( Vec u ) bytes → !v String

Payload bytes for the current tensor, any chunking. When a tensor's declared size is reached the writer advances to the next declared tensor (inserting the alignment gap first). Overrun is an error.

@ gws_finish * GgufS s → !v String

All payloads in? Flush and close. The handle is closed here (not in gws_free) so the caller sees the failure, not a silent short file.

@ gws_free * GgufS s → v

---

main.nu

gguf — inspect, verify and export GGUF model containers.

gguf info <file> one-line summary gguf dump <file> header + all metadata + tensors gguf kv <file> <key> print one metadata value gguf tensors <file> tensor table gguf verify <file> deep structural check (+ overlap) gguf export <file> <tensor> -o out.f32 dequantise to f32-LE gguf gen <out.gguf> write the reference sample file gguf selftest write → parse → compare, bit-exact

The parser treats every input as hostile: malformed, truncated or adversarial files produce an error message, never a crash, hang or unbounded allocation (see tests/gguf_test.sh, which proves this with a corruption battery).

API

: i GGUF_CLI_VERSION_MAJOR 0

: STCnt

: STCnt {
    i pass
    i fail
}

@ string_eq_raw String a s raw → b

String vs raw-literal equality (helper for selftest)

@ main → i

---

dequant.nu

packages/gguf/src/dequant.nu — host-side scalar dequantisation.

Reference implementation of the ggml block-quant decodings, pure NURL, no GPU dependency: this is the CPU decode path AND the golden oracle a GPU dequant kernel is verified against. Output is the upload-ready canonical form — a ( Vec u ) of little-endian IEEE-754 f32 values, one per tensor element, in storage order.

( gg_f16_to_f32_bits h ) → i IEEE half → f32 bit pattern ( gg_f16_to_f ) → f IEEE half → double ( gguf_dequant g idx ) → !( Vec u ) String f32-LE buffer ( gguf_dequant_f64 g idx ) → !( Vec f ) String convenience

Supported source types: F32, F64, F16, BF16, Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, and the K-quants Q4_K / Q5_K / Q6_K (the formats modern llama.cpp models actually ship in — a Q4_K_M file mixes Q4_K and Q6_K tensors). Anything else is a clean error naming the type — never a wrong answer. Layouts follow ggml exactly: Q4_0: 18-byte block = f16 scale d + 16 nibble bytes; elements 0..15 are the LOW nibbles, 16..31 the HIGH nibbles; value = d (q - 8). Q4_1: 20-byte block = f16 d + f16 min m + 16 nibble bytes; value = d q + m (same nibble order). Q8_0: 34-byte block = f16 d + 32 signed bytes; value = d * q.

API

@ gg_f16_to_f32_bits i h → i

IEEE-754 binary16 → binary32 — the stdlib bit-transport primitive (stdlib/std/floatbits.nu), kept under the historical gg_ name for the package's public dequant API.

@ gg_f16_to_f i h → f

@ gguf_dequant_range * Gguf g i idx i first i count → !( Vec u ) String

Dequantise a contiguous ELEMENT RANGE of tensor #idx — [first, first+count) — to little-endian f32. first and count must be multiples of the type's block size (they always are for whole rows: ggml guarantees dim0 is a block multiple). This is what lets a consumer touch one row of a multi-gigabyte embedding table without expanding the whole tensor: nurllama dequantises exactly the token's row per step.

@ gguf_dequant * Gguf g i idx → !( Vec u ) String

Dequantise the whole tensor.

@ gguf_dequant_f64 * Gguf g i idx → !( Vec f ) String

Convenience: dequantise to host doubles (tests, small tensors).