gguf.nu

packages/gguf/src/gguf.nu — a bulletproof GGUF container parser.

GGUF is the ggml-ecosystem tensor container (llama.cpp, whisper.cpp, stable-diffusion.cpp): a little-endian header, typed metadata key/values, a tensor table, then an aligned data section. This module parses versions 2 and 3, mmap-backed, without ever loading tensor data into memory — tensor bytes are addressed lazily straight out of the mapping.

Trust model: the file is UNTRUSTED input. Every count, length and offset is validated against the actual file size BEFORE any allocation or read depends on it; allocations are proportional to bytes actually consumed from the file, never to what a header merely claims. A malformed or truncated file is an error, never a crash, a hang, or an unbounded allocation.

( gguf_open path ) → !Gguf String mmap + parse ( gguf_parse_bytes data ) → !Gguf String parse a buffer (BORROWS data — keep it alive until gguf_close) ( gguf_close g ) → v unmap + free ( gguf_n_kv g ) ( gguf_n_tensors g ) → i ( gguf_find_kv g key ) → i index, -1 when absent ( gguf_kv_int_or g key def ) → i any int/bool-typed KV ( gguf_kv_f_or g key def ) → f f32/f64-typed KV ( gguf_kv_str_or g key def ) → s BORROWED from g ( gguf_find_tensor g name ) → i index, -1 when absent ( gguf_tensor_ptr g t ) → *u BORROWED tensor bytes ( gguf_type_name t ) ( gguf_type_blck t ) ( gguf_type_size t )

Layout invariants enforced at parse time: magic, version ∈ {2,3}, sane KV/tensor counts, in-bounds strings, no nested arrays, unique KV keys and tensor names, power-of-two alignment, dims ≥ 1 with overflow-checked element counts, aligned tensor offsets, and every sizable tensor fully inside the data section.

API

: i GGUF_VT_U8 0

── GGUF metadata value types (spec) ────────────────────────────────

: i GGUF_VT_I8 1

: i GGUF_VT_U16 2

: i GGUF_VT_I16 3

: i GGUF_VT_U32 4

: i GGUF_VT_I32 5

: i GGUF_VT_F32 6

: i GGUF_VT_BOOL 7

: i GGUF_VT_STR 8

: i GGUF_VT_ARR 9

: i GGUF_VT_U64 10

: i GGUF_VT_I64 11

: i GGUF_VT_F64 12

@ gguf_type_name i t → s

── ggml tensor types ─────────────────────────────────────────────── Name for a ggml type id; ? for ids this parser has no name for.

@ gguf_type_blck i t → i

Elements per quantisation block; 0 when this parser cannot size the type (unknown or not-yet-tabled id — such tensors are still listed, but their byte extent cannot be verified or dequantised).

@ gguf_type_size i t → i

Bytes per quantisation block; 0 when unknown (pairs with gguf_type_blck).

: GgufKv

: GgufKv {
    String key
    i vt
    i ival
    f fval
    String sval
    i atype
    ( Vec i ) ai
    ( Vec f ) af
    ( Vec String ) astr
}

One metadata key/value. vt discriminates which payload field holds the value: int family (u8..u64/i64/bool) → ival, f32/f64 → fval, string → sval, array → atype + one of ai/af/astr.

: GgufTensor

: GgufTensor {
    String name
    i gtype
    i nd
    i d0
    i d1
    i d2
    i d3
    i nelems
    i offset
    i nbytes
}

One tensor-table entry. offset is RELATIVE to the data section (exactly as stored in the file); absolute bytes are reached through gguf_tensor_ptr. nbytes is -1 when the type cannot be sized.

: Gguf

: Gguf {
    i version
    i align
    ( Vec GgufKv ) kvs
    ( Vec GgufTensor ) tensors
    i data_off
    i data_size
    * u map
    i map_size
    b from_mmap
    ( Vec u ) buf
}

: GCur

: GCur {
    * u p
    i n
    i off
    b fail
}

── bounded little-endian cursor over the raw mapping ─────────────── Every read checks the remaining byte budget first; the first out-of-bounds read latches fail and all subsequent reads return zero values, so a parse can run to a checkpoint and test fail once.

@ gguf_open s path → !*Gguf String

mmap-backed open (POSIX). Platforms without MAP_PRIVATE (wasm, win32) fall back to reading the whole file into an owned buffer — correct everywhere, mmap-lazy where it matters.

@ gguf_parse_bytes ( Vec u ) data → !*Gguf String

Parse an in-memory GGUF image. BORROWS data — the caller keeps the vec alive until gguf_close and frees it afterwards.

@ gguf_close * Gguf g → v

@ gguf_version * Gguf g → i

@ gguf_align * Gguf g → i

@ gguf_n_kv * Gguf g → i

@ gguf_n_tensors * Gguf g → i

@ gguf_data_size * Gguf g → i

@ gguf_find_kv * Gguf g s key → i

@ gguf_kv_int_or * Gguf g s key i def → i

@ gguf_kv_f_or * Gguf g s key f def → f

@ gguf_kv_str_or * Gguf g s key s def → s

BORROWED: the returned s points into g (or is def); valid until gguf_close. Do not free.

@ gguf_find_tensor * Gguf g s name → i

@ gguf_tensor_ptr * Gguf g GgufTensor t → *u

BORROWED pointer to a tensor's first byte inside the mapping; valid until gguf_close. Length is t.nbytes (when ≥ 0).

---

write.nu

packages/gguf/src/write.nu — a GGUF v3 writer/builder.

The reader's round-trip partner: build typed metadata and tensors in memory, then serialise a spec-exact GGUF v3 image. Powers the test suite (write → parse → compare, bit for bit) and gives the ecosystem an export path (tensor snapshots, converted models, fixtures).

( gw_new align ) → !*GgufW String ( gw_kv_u32 w key v ) ( gw_kv_i32 … ) ( gw_kv_u64 … ) ( gw_kv_f32 w key x ) ( gw_kv_f64 … ) ( gw_kv_bool … ) ( gw_kv_str w key val ) ( gw_kv_arr_i32 w key vals ) — ( Vec i ), borrowed ( gw_kv_arr_f32 w key vals ) — ( Vec f ), borrowed ( gw_kv_arr_str w key vals ) — ( Vec String ), borrowed ( gw_tensor w name gt nd d0 d1 d2 d3 bytes ) → !v String ( gw_finish w ) → ( Vec u ) the file image ( gw_write w path ) → !v String ( gw_free w )

gw_new emits general.alignment itself (so the image is self-describing) — callers must not add that key again. gw_tensor validates the payload length against the declared type/dims and pads the data section to the alignment, exactly as the parser will demand on the way back in.

API

: GgufW

: GgufW {
    i align
    i n_kv
    ( Vec u ) kvb
    ( Vec String ) tnames
    ( Vec i ) ttype
    ( Vec i ) tnd
    ( Vec i ) td0
    ( Vec i ) td1
    ( Vec i ) td2
    ( Vec i ) td3
    ( Vec i ) toff
    ( Vec u ) data
}

@ gw_new i align → !*GgufW String

@ gw_kv_u32 * GgufW w s key i v → v

@ gw_kv_i32 * GgufW w s key i v → v

@ gw_kv_u64 * GgufW w s key i v → v

@ gw_kv_i64 * GgufW w s key i v → v

@ gw_kv_f32 * GgufW w s key f x → v

@ gw_kv_f64 * GgufW w s key f x → v

@ gw_kv_bool * GgufW w s key b v → v

@ gw_kv_str * GgufW w s key s val → v

@ gw_kv_arr_i32 * GgufW w s key ( Vec i ) vals → v

@ gw_kv_arr_f32 * GgufW w s key ( Vec f ) vals → v

@ gw_kv_arr_str * GgufW w s key ( Vec String ) vals → v

@ gw_tensor * GgufW w s name i gt i nd i d0 i d1 i d2 i d3 ( Vec u ) bytes → !v String

Add a tensor: validates the declared shape against the payload size with the same rules the parser enforces, aligns the data section, and records the entry. Unused trailing dims pass 1.

@ gw_finish * GgufW w → ( Vec u )

Serialise the complete GGUF v3 image.

@ gw_write * GgufW w s path → !v String

@ gw_free * GgufW w → v

---

main.nu

gguf — inspect, verify and export GGUF model containers.

gguf info <file> one-line summary gguf dump <file> header + all metadata + tensors gguf kv <file> <key> print one metadata value gguf tensors <file> tensor table gguf verify <file> deep structural check (+ overlap) gguf export <file> <tensor> -o out.f32 dequantise to f32-LE gguf gen <out.gguf> write the reference sample file gguf selftest write → parse → compare, bit-exact

The parser treats every input as hostile: malformed, truncated or adversarial files produce an error message, never a crash, hang or unbounded allocation (see tests/gguf_test.sh, which proves this with a corruption battery).

API

: i GGUF_CLI_VERSION_MAJOR 0

: STCnt

: STCnt {
    i pass
    i fail
}

@ string_eq_raw String a s raw → b

String vs raw-literal equality (helper for selftest)

@ main → i

---

dequant.nu

packages/gguf/src/dequant.nu — host-side scalar dequantisation.

Reference implementation of the ggml block-quant decodings, pure NURL, no GPU dependency: this is the CPU decode path AND the golden oracle a GPU dequant kernel is verified against. Output is the upload-ready canonical form — a ( Vec u ) of little-endian IEEE-754 f32 values, one per tensor element, in storage order.

( gg_f16_to_f32_bits h ) → i IEEE half → f32 bit pattern ( gg_f16_to_f ) → f IEEE half → double ( gguf_dequant g idx ) → !( Vec u ) String f32-LE buffer ( gguf_dequant_f64 g idx ) → !( Vec f ) String convenience

Supported source types: F32, F64, F16, BF16, Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, and the K-quants Q4_K / Q5_K / Q6_K (the formats modern llama.cpp models actually ship in — a Q4_K_M file mixes Q4_K and Q6_K tensors). Anything else is a clean error naming the type — never a wrong answer. Layouts follow ggml exactly: Q4_0: 18-byte block = f16 scale d + 16 nibble bytes; elements 0..15 are the LOW nibbles, 16..31 the HIGH nibbles; value = d (q - 8). Q4_1: 20-byte block = f16 d + f16 min m + 16 nibble bytes; value = d q + m (same nibble order). Q8_0: 34-byte block = f16 d + 32 signed bytes; value = d * q.

API

@ gg_f16_to_f32_bits i h → i

IEEE-754 binary16 → binary32 — the stdlib bit-transport primitive (stdlib/std/floatbits.nu), kept under the historical gg_ name for the package's public dequant API.

@ gg_f16_to_f i h → f

@ gguf_dequant_range * Gguf g i idx i first i count → !( Vec u ) String

Dequantise a contiguous ELEMENT RANGE of tensor #idx — [first, first+count) — to little-endian f32. first and count must be multiples of the type's block size (they always are for whole rows: ggml guarantees dim0 is a block multiple). This is what lets a consumer touch one row of a multi-gigabyte embedding table without expanding the whole tensor: nurllama dequantises exactly the token's row per step.

@ gguf_dequant * Gguf g i idx → !( Vec u ) String

Dequantise the whole tensor.

@ gguf_dequant_f64 * Gguf g i idx → !( Vec f ) String

Convenience: dequantise to host doubles (tests, small tensors).