NURLNURL registrynurl-lang.org →

← yoloe

yoloe 0.1.0 API

decode.nu

packages/yoloe/src/decode.nu — turn YOLOE's output0 into boxes.

output0 is [1, 4+nc+nm, na] (na = 8400 anchors over 3 scales): the first 4 channels are the box (cx,cy,w,h, already decoded to pixels in the letterboxed frame), the next nc are per-class scores (already sigmoid), the rest are segmentation mask coefficients (ignored here). For each anchor we take the best class above a threshold, then non-max suppress.

API

: Detection { i cls f score f cx f cy f w f h }

A detection in letterboxed-pixel centre/size coordinates.

& c @ nurl_peek_f32 *u base i idx → f

@ yolo_decode *u out i na i nc f thresh → ( Vec Detection )

Decode the best detections above thresh. nc = number of classes, na = anchors, no = total channels (4 + nc + masks).

@ yolo_nms ( Vec Detection ) dets f iou_thresh → ( Vec Detection )

Greedy non-max suppression (per class), keeping highest-score boxes.

image.nu

packages/yoloe/src/image.nu — image I/O + YOLO letterbox preprocessing.

Reads/writes binary PPM (P6), letterboxes to the model's square input (aspect-preserving resize + grey pad), packs to an NCHW float tensor normalised to [0,1], and draws labelled boxes. An Image keeps the raw file bytes plus a pixel-data offset, so reading copies nothing; a fresh image (letterbox/blank) owns a new byte buffer.

API

& c @ nurl_poke_f32 *u base i idx f val → v

: Image { i w i h ( Vec u ) buf i off }

: Letterbox { Image img f scale i padx i pady }

Letterbox result: the square image plus the transform back to original pixels (orig = (lb - pad) / scale).

@ img_w Image im → i

@ img_h Image im → i

@ img_get Image im i x i y i ch → i

@ img_set Image im i x i y i ch i val → v

@ ppm_read s path → ?Image

@ img_blank i w i h i fill → Image

@ ppm_write s path Image im → b

@ letterbox Image im i S → Letterbox

── letterbox to S×S (aspect-preserving, grey 114 pad) ────────────

@ img_to_nchw_norm Image im → *u

Pack to NCHW float tensor normalised to [0,1].

@ img_draw_rect Image im i x0 i y0 i x1 i y1 i r i gg i bb → v

prompt.nu

packages/yoloe/src/prompt.nu — RUNTIME-promptable open-vocabulary detection.

yoloe-prompt <model.onnx> <tpe.f32> <classes.txt> <image.ppm> [out.ppm]

Unlike src/main.nu (whose vocabulary is baked into the model at export time), this takes the prompt text embeddings as a runtime input: the model has two inputs, the image and tpe [1,K,512] (raw MobileCLIP text features for the K prompt words). Swap the tpe file + classes file to detect a different set of objects with the same model — no re-export.

Produce the model + tpe with tools/export_promptable.py and tools/gen_tpe.py.

API

& c @ nurl_peek_f32 *u base i idx → f

@ p s m → v

@ pf f x → v

@ pn i n → v

@ load_f32 s path *u pcell → *u

@ read_names s path → ( Vec String )

@ name_at ( Vec String ) names i i → s

@ shape4 i a i b i c i d → ( Vec i )

@ shape3 i a i b i c → ( Vec i )

@ main → i

bpe.nu

packages/yoloe/src/bpe.nu — the CLIP byte-pair-encoding tokenizer in pure NURL. This is the word → token-id half of the MobileCLIP text path; the other half (token-ids → 512-d features) is the text-encoder ONNX graph already run by the onnx runtime (see tests/text_fwd.nu).

Faithful port of open_clip's SimpleTokenizer (the tokenizer MobileCLIP's ClipTokenizer wraps as get_tokenizer("ViT-B-16")):

bpe_simple_vocab_16e6.txt, lines 1..48894),

Word splitting reproduces CLIP's regex 'r"'s|'t|'re|'ve|'m|'ll|'d|[\p{L}]+|[\p{N}]|[^\s\p{L}\p{N}]+" for the common case (ASCII / Latin-1 letters, digits, punctuation, English contractions). A letter is a-z / Latin-1 letters / any codepoint

= U+0100; a digit is 0-9. Exotic symbol-vs-letter classification of rare

BMP punctuation may differ from full Unicode \p{L}/\p{N}, which does not arise for object-detection class names.

API

: i BPE_INF 1000000000

: Tokenizer

: Tokenizer {
    ( Vec String ) byte_enc      // byte value 0..255 → its symbol string
    ( HashMap s i ) enc          // token string → id
    ( HashMap s i ) ranks        // "a b" merge line → rank
    ( Vec String ) arena         // owns enc/ranks key strings
    i sot
    i eot
}

A loaded tokenizer. The hash maps key on s (raw char*) pointers, so the backing strings must outlive the maps: arena owns every key string that is not already owned by byte_enc.

@ tokenizer_load s merges_path → Tokenizer

── load ───────────────────────────────────────────────────────── Build the tokenizer from the merges asset. Returns a Tokenizer whose maps are empty (sot=-1) on read failure.

@ bpe_encode Tokenizer tk s text → ( Vec i )

── encode / tokenize ──────────────────────────────────────────── text → list of token ids (no sot/eot, no padding).

@ bpe_tokenize Tokenizer tk s text i ctx → ( Vec i )

text → padded context: [sot] + ids + [eot], zero-padded/truncated to ctx. Returns a Vec<i> of length ctx.

main.nu

packages/yoloe/src/main.nu — promptable open-vocabulary detection.

yoloe <model.onnx> <classes.txt> <image.ppm> [out.ppm]

Runs an exported YOLOE detector on the GPU through the onnx package, decodes the boxes, suppresses overlaps, prints the labelled detections, and writes an annotated image. The class vocabulary is read from a text file (one prompt per line) — it must match the names the model was exported with (tools/export.py), so swapping both detects a different vocabulary. The detector itself is vocabulary-agnostic.

Build: nurlpkg install then NURL_STDLIB=<repo> ../../nurl.sh src/main.nu

API

& c @ nurl_peek_f32 *u base i idx → f

@ p s m → v

@ pf f x → v

@ pn i n → v

@ read_names s path → ( Vec String )

Read the prompt vocabulary from a text file (one class per line); the number of lines is the class count nc. The file must match the names the model was exported with — swap both to detect a different vocabulary.

@ name_at ( Vec String ) names i i → s

@ shape4 i a i b i c i d → ( Vec i )

@ main → i