NURL registrynurl-lang.org →packages/whisper/src/kernels.nu — the encoder/decoder kernels.
Whisper is not the llama shape, and every kernel here is a place it differs:
ACT2FN["gelu"] is the exact one; the tanh form is a different function and using it would be a quiet, small, everywhere-wrong bias.
which is what turns 3000 spectrogram frames into 1500 encoder positions.
future; the whisper encoder looks at the whole 30 seconds at once, and the decoder's cross-attention looks at all of it too.
Every kernel is a CHECKED kernel (packages/gpukit): it reaches device memory only through LD(name, i) / ST(name, i, v) over the buffers its launcher binds, and every index is checked against that buffer's length. A launcher takes the buffers themselves — GkBuf, and GkHBuf for weights kept as 16-bit halves — never their addresses, so a wrong size or offset can give a wrong number but never a read or write outside the model's memory, and every launcher is a plain safe function. The same kernels run on gpukit's CPU backend.
A launcher RECORDS its launch and appends it to a plan (a Vec GkLaunch) rather than making it: the model records its encoder, the cross-attention projections and one decoder step when it opens, and replays them — a replay is one driver call per kernel, with no binding and no checking. What changes from one decoder step to the next (the token, the position) is a scalar the replay is given (wk_set / wk_attn_keys).
: WhW { GkBuf f GkHBuf h }A matrix weight on the device: f32 (f), or the checkpoint's own 16-bit halves (h). Halves are widened at the point of use — exactly, denormals included — so the arithmetic is f32 either way and the result is bit-identical; what changes is that the weight bytes crossing the memory bus of a memory-bound matvec are half. Exactly one of the two is there.
@ wk_w_none → WhW@ wk_w_ok WhW w → b@ wk_run ( Vec GkLaunch ) plan → bReplay every launch of plan, in order. F on a device error.
@ wk_set inout ( Vec GkLaunch ) plan i at s name i x → bGive scalar name of the plan's launch at the value x (a decoder step's position or token).
@ wk_matvec GpuKit kit b warp WhW w GkBuf x GkBuf bias GkBuf y i rows i cols i batch inout ( Vec GkLaunch ) plan → by[batch][rows] = W[rows][cols] · x[batch][cols] + bias[rows]. bias may be none (gk_buf_none). warp = the CUDA shapes: a 64x64-tiled GEMM for many positions (the encoder), a warp per row and eight positions for a few, a warp per row for one (the decoder); without it, the portable kernel.
@ wk_matvec_ex GpuKit kit b warp WhW w GkBuf x GkBuf bias GkBuf y i yoff b gelu GkBuf add i rows i cols i batch inout ( Vec GkLaunch ) plan → bThe same, its output written at y[yoff + …] (a decoder step's k and v go straight into their row of the KV cache), through the exact GELU when gelu, plus add (same shape as the output) when it is not none.
@ wk_layernorm GpuKit kit b warp GkBuf x GkBuf res GkBuf w GkBuf bias GkBuf y i n f eps i batch inout ( Vec GkLaunch ) plan → by = LayerNorm(x) over rows of n; with res (not none) x += res first, in place.
@ wk_conv1d_gelu GpuKit kit b warp GkBuf x GkBuf w GkBuf bias GkBuf add GkBuf y i t_in i c_in i c_out i stride i pad inout ( Vec GkLaunch ) plan → b@ wk_attn GpuKit kit b warp GkBuf q GkBuf k GkBuf v GkBuf sc GkBuf out i nh i hd i nq i nkey f qscale i causal inout ( Vec GkLaunch ) plan → bAttention: out[nq][nh·hd] from q[nq][nh·hd] and K, V [nkey][nh·hd]. sc is scratch — the one-query path's per-chunk partials (nh·nq·32·66 floats), or the composed path's score matrix (nh·nq·nkey); the fused many-query path needs none (it may be none then). Appends one or two launches.
@ wk_attn_keys inout ( Vec GkLaunch ) plan i at b warp i nh i hd i nq i nkey i causal → bThe attention wk_attn recorded at plan[at] (with the same warp, hd, nq and causal), now over nkey keys — the decoder's self-attention grows by one key a step.
@ wk_addv GpuKit kit GkBuf a GkBuf b i off i n inout ( Vec GkLaunch ) plan → ba[i] += b[off + i] for i < n. off 0 is a plain add; the decoder adds its position's row of the positional embedding by offset.
@ wk_embed GpuKit kit WhW table GkBuf pos GkBuf y i row i off i n inout ( Vec GkLaunch ) plan → bA token's row of the embedding table plus its position's row of the positional embedding → the decoder's activation. The table is on the device (it is also the output projection — whisper ties them), so a token's row is a copy, not a host round-trip.
@ wk_cvt GpuKit kit GkHBuf src GkBuf dst i n inout ( Vec GkLaunch ) plan → bWiden 16-bit weights (f16, or bf16 — the GkHBuf says which) to f32 on the device: half the bytes over PCIe, and the widening happens where there are thousands of threads for it instead of one host loop doing 378 million iterations.
@ wk_argmax GpuKit kit GkBuf x i n GkBuf out inout ( Vec GkLaunch ) plan → b── Greedy decoding's argmax, on the device ─────────────────────────
The only thing greedy decoding wants from a step is WHICH logit is largest. Fetching the row to find out cost a 51865-float transfer and a 51865-iteration host loop — 365 us a token against a 477 us decode step. Reduced where it lives, the answer is one integer (out[0], an i64 buffer). One block of 1024 threads: each scans its stride of the row, then the warps fold their 32 candidates with shuffles and the 32 warp winners fold the same way — two shared-memory round trips, not a barrier per halving. Ties go to the LOWER index at every step, which is what the host scan's > gives — so the token stream is unchanged, not merely equivalent.
packages/whisper/src/main.nu — the CLI (encoder stage).
whisper transcribe <model> <audio.wav> [--lang en] [--max N] whisper encode <config.json> <model.safetensors> <audio.wav> -o enc.f32
transcribe is the whole thing: WAV → log-mel → encoder → decoder → text. encode stops after the encoder and writes its 1500 × d_model states, which is what the test suite compares against HF's WhisperModel.encoder.
<model> is a local directory (config.json, model.safetensors, tokenizer.json side by side), a whisper.cpp ggml file, OR a Hugging Face ref (e.g. openai/whisper-base) — a ref is fetched into ~/.nurl/models via the hub package and the downloaded path is used.
@ main → ipackages/whisper/src/serve.nu — whisper as a service.
whisper serve <model-dir> --addr 0.0.0.0:6543 [--lang en] [--vad] [--timestamps]
The point of a server is WHEN the model loads. The CLI pays the whole bill per invocation — read 1.5 GB of safetensors, convert f16 to f32 on the device, compile the kernels — and only then hears the audio. Here all of that happens once, before the port opens; a request pays for its own audio and nothing else.
The HTTP surface is whisper.cpp's, so its clients work unchanged:
POST /inference multipart/form-data with a file field (WAV), plus optional language and response_format (json|text) fields → {"text":"..."} or the bare text. A raw WAV body (no multipart) is also accepted — curl --data-binary @clip.wav is not a browser form, and there is no reason to make it pretend to be one. GET /health {"status":"ok", model shape, request count} GET / the same (a browser poking the port should learn something, not get a 404)
Per-request vad and timestamps fields override the server-side flags, so one server can serve both a subtitle pipeline and a bare-text one.
Handlers are top-level functions reading module globals, not closures over locals — the same idiom as nurllama's api.nu, and for the same reason: closure environments are manual in NURL, and a server's handlers live for the process.
: ~ b g_srv_vad F: ~ b g_srv_ts F: ~ i g_srv_max 0: ~ f g_srv_nospeech 0.6: SrvCfg: SrvCfg {
String dir
String lang
String token
}The server's text settings: the model argument (kept for the reloads), the default language, and the bearer token (empty = open server, the loopback default). Set before the port opens; requests and the unload reaper read copies. (As string globals, the compiler cannot see that nothing replaces them while another thread reads them, and reset at the end they would be.)
: ( Locked SrvCfg ) g_srv_cfg ( __srv_cfg0 ): SrvState: SrvState {
? Whisper w // the loaded model, OWNED by the server (F = none loaded)
i reqs
i busy // requests + streams holding the model right now
i idle_since // monotonic_ns when the last holder let go
i loads // (re)loads so far — 1 is the load before the port opened
i unloads
i load_ms // the last load's wall time
i unload_ms // 0 = the model stays for the process's life
b gpu // a transcription is running (one at a time: see __srv_gpu_try)
}── the model as a lease: loaded on demand, released when idle ──────
--unload-after N makes the model a thing the server HOLDS rather than IS. A reaper thread watches the clock; N seconds after the last request ends, with nothing in flight, it closes the model — device buffers, kernels, the CUDA context, the tokenizer stays (a few MB) — and the next request reloads it before it runs. What the server keeps while idle is the port and about a hundred megabytes of process; what it gives back is everything the model was (large-v3: 4.8 GB of device memory).
Who may touch the model: a request holds busy from acquire to release, a WebSocket stream for the life of the connection, and the reaper only ever closes a model nobody holds. All of it is one Locked record set at start — reached from every handler and from the reaper in safe code, and held by nothing longer than a load.
: ( Locked SrvState ) g_srv ( __srv_blank ): ( Locked ? Tok ) g_srv_tok ( __srv_tok_blank ) // one owner of the tokenizer, kept while serving: WsCfg: WsCfg {
b f32 // the client sends float32 frames, else PCM16
String lang // the language override ("" = the server's default)
WssCfg st // the endpointer's settings
b silero // F = the energy detector, by request
}One stream's settings, from its config messages. They belong to the connection: as globals, two streams open at once took each other's sample format and language (and one could free the language string the other was transcribing with).
: ( Locked ? Silero ) g_srv_vad_net ( __srv_vad_blank ): WsDrafts: WsDrafts {
( Vec u ) last
( Vec u ) full
i full_seg
// a stream's periodic drafts take at most half the GPU's time: after
// one that took D ms, the next waits until the stream is 2·D further on
i next_ok
}One stream's drafts so far: the last one (what stable is agreed against) and the last full-context one (which can be the final).
@ wh_serve s dir s host i port s lang i maxtok b use_vad b with_ts s cert s key s token i unload_s s vad_model b energy → iLoad the model, open the port, serve until stopped.
packages/whisper/src/model.nu — the encoder.
Whisper's encoder turns a 30-second log-mel spectrogram (3000 frames × 80 mels) into 1500 vectors the decoder attends to. The shape, read out of the checkpoint's own tensor names rather than remembered:
conv1 (k=3, s=1, pad=1) 80 → d_model then GELU conv2 (k=3, s=2, pad=1) d_model → d_model then GELU 3000 → 1500 frames
N × [ LN → self-attn (NON-CAUSAL) → +residual LN → fc1 → GELU → fc2 → +residual ] LN
The corner that bites: k_proj HAS NO BIAS. q, v and out_proj all have one; k does not. Whisper is not alone in this — it is inherited from the original implementation — and a loader that adds a zero bias where none exists is fine, while one that expects a bias tensor and finds none will happily read whatever is next in the file.
Device memory is gpukit's: every tensor and scratch buffer is a GkBuf (a GkHBuf for a matrix kept in the checkpoint's own halves) — a view into one of the model's 128 MB arena chunks that holds its chunk alive — and every kernel is a checked one (kernels.nu). Nothing here handles a device address.
: WhEnc: WhEnc {
GkBuf ln1_w
GkBuf ln1_b
WhW wq
GkBuf bq
WhW wk
WhW wv
GkBuf bv
WhW wo
GkBuf bo
GkBuf ln2_w
GkBuf ln2_b
WhW fc1_w
GkBuf fc1_b
WhW fc2_w
GkBuf fc2_b
}One encoder layer's weights. k_proj has no bias — none is looked for.
: WhDec: WhDec {
GkBuf ln1_w
GkBuf ln1_b
WhW wq
GkBuf bq
WhW wk // no bias, like the encoder's
WhW wv
GkBuf bv
WhW wo
GkBuf bo
GkBuf lnx_w // cross-attention's norm
GkBuf lnx_b
WhW xwq
GkBuf xbq
WhW xwk // no bias
WhW xwv
GkBuf xbv
WhW xwo
GkBuf xbo
GkBuf ln2_w
GkBuf ln2_b
WhW fc1_w
GkBuf fc1_b
WhW fc2_w
GkBuf fc2_b
GkBuf kcache // n_ctx_dec × d_model
GkBuf vcache
GkBuf xk // the encoder's K for cross-attention, computed ONCE per clip
GkBuf xv
}One decoder layer: its weights, and its caches.
: WhisperImpl: WhisperImpl {
GpuKit kit
b warp // the CUDA kernel shapes (warp shuffles, shared-memory tiles)
b w_half // matrix weights live on the device as raw f16 halves
? St st // the safetensors file (HF checkpoint) — F in ggml mode
? Gg gg // whisper.cpp's legacy ggml container — F in HF mode
// the load's upload queue: every tensor is allocated as it is met and
// sent in ONE streamed batch (gk_upload_batch) once the last one is
// known; f16/bf16 tensors that must be widened wait in cvt_* for their
// bytes to land, then a kernel widens each and the raw arena goes back
( Vec GkBuf ) up_dst // f32 destinations
( Vec i ) up_ti // each one's index in the container
( Vec GkHBuf ) uph_dst // 16-bit destinations (half matrices, widen sources)
( Vec i ) uph_ti
// the weight arena: device memory is carved from 128 MB chunks, so a
// model is ~40 allocations rather than 1,300 — and ~40 frees when it is
// closed, which is what an idle unload pays. A view holds its chunk.
GkBuf arena // the chunk being carved (none = no chunk yet)
i arena_off // bytes
// the raw arena: f16/bf16 sources waiting to be widened live here until
// the widen has run
GkBuf raw
i raw_off
( Vec GkHBuf ) cvt_src // per tensor: its halves …
( Vec GkBuf ) cvt_dst // … and the f32 buffer the model will use
i n_mels
i d_model
i e_ff // the encoder's FFN width (4·d_model in every released whisper)
i d_ff // the decoder's
String bad // the first tensor whose size disagreed with the config
i n_head
i head_dim
i d_head // the DECODER's head count — the same in every
i d_head_dim // whisper so far, but the config states both
i n_enc_layer
i n_dec_layer
i n_ctx_enc // 1500 encoder positions
i n_vocab
// encoder weights
GkBuf conv1_w
GkBuf conv1_b
GkBuf conv2_w
GkBuf conv2_b
GkBuf enc_pos
( Vec WhEnc ) enc
GkBuf e_lnf_w
GkBuf e_lnf_b
// decoder weights
WhW tok_embd // also the output projection — whisper ties them
GkBuf dec_pos
( Vec WhDec ) dec
GkBuf d_lnf_w
GkBuf d_lnf_b
i n_ctx_dec
// decoder scratch: one token's activations
GkBuf dx_d
GkBuf dxn_d
GkBuf dq_d
GkBuf dao_d
GkBuf dtmp_d
GkBuf dff_d
GkBuf dsc_d // attention scratch: the composed path's score row, or the
// one-query fused path's per-chunk partials
GkBuf logits_d
GkBuf argmax_d // one i64: greedy decoding's answer
b oom // a device allocation came back empty — see __wh_scratch
// encoder scratch
GkBuf mel_d // 3000 × n_mels
GkBuf c1_d // 3000 × d_model
GkBuf c2_d // 1500 × d_model
GkBuf xn_d
GkBuf q_d
GkBuf k_d
GkBuf v_d
GkBuf ao_d
GkBuf tmp_d
GkBuf ff_d // 1500 × 4·d_model
GkBuf sc_d // n_head × 1500 × 1500 scores (the composed attention only)
GkBuf enc_out // 1500 × d_model — what the decoder attends to
// the recorded launches (kernels.nu): checked and bound once, replayed
( Vec GkLaunch ) enc_plan // the encoder, for enc_frames mel frames
i enc_frames // = 2 × the positions the encoder and cross-attention run over
( Vec GkLaunch ) cross_plan // the decoder's cross-attention K and V
( Vec GkLaunch ) dec_plan // one decoder step
( Vec i ) dec_at // per decoder layer: its KV-cache writes and self-attention
( Vec i ) dec_xat // per decoder layer: its cross-attention
i dec_embed // the token's and the position's embedding rows
( Vec GkLaunch ) am_plan // the argmax
}: WhInfo: WhInfo {
i n_mels
i d_model
i n_ctx_enc
i n_enc_layer
i n_dec_layer
}What a caller sizes inputs by: fixed once the model is open.
: Whisper: Whisper {
( Locked WhisperImpl ) l
WhInfo info
? Gg gg // the ggml container the model came from (F in HF mode)
}A Whisper is a handle: every copy is the same model, and the last owner releases it — every device allocation, the kernels, the context, the weight file. The engine is Locked: one encode, decode step or load at a time; its shape and the container it came from are shared beside it. wh_close is an optional early release.
@ Whisper_share Whisper h → Whisper@ Whisper_drop sink Whisper h → v@ wh_n_mels Whisper h → iThe model's shape, for callers that size inputs.
@ wh_d_model Whisper h → i@ wh_n_ctx_enc Whisper h → i@ wh_n_enc_layer Whisper h → i@ wh_n_dec_layer Whisper h → i@ wh_gg Whisper h → ?Ggthe ggml container the model came from (F in HF mode)
@ wh_bind_thread Whisper h → bBind the model's device to the calling thread (a CUDA context is per-thread).
@ wh_open s config_path s weights_path → !Whisper String@ wh_open_ggml s path → !Whisper StringOpen a whisper.cpp ggml container: hyperparameters and tokenizer live in the same file as the weights, so the only argument is the file.
@ wh_close sink Whisper w → vEarly release (optional): the last owner gives back every device allocation, the kernels, the context and the weight file.
@ wh_encode Whisper h ( Vec f ) mel → vRun the encoder over a 30-second log-mel spectrogram: mel is 3000 × n_mels, row-major (a frame is contiguous) — exactly what packages/audio produces. Leaves the 1500 encoder states on the device (wh_enc_out).
@ wh_enc_out Whisper h → ( Vec f )The encoder states, back on the host: 1500 × d_model, row-major.
@ wh_prepare_cross Whisper h → vCross-attention's K and V for the clip just encoded.
@ wh_decode_step Whisper h i tok i pos → vOne decoder step: token tok at position pos, with everything before it already in the KV cache. Leaves the logits on the device.
@ wk_addv_row Whisper h i pos → vx += embed_positions[pos], on its own: the step's own addition of its position, for a caller driving the decoder by hand.
@ wh_logits Whisper h → ( Vec f )The logits of the last step, on the host.
@ wh_argmax_dev Whisper h → iThe greedy pick, without fetching the logits: the reduction runs on the device and one integer comes back. Ties go to the lower index in the kernel exactly as they do in wh_argmax below, so the token stream is the same one.
@ wh_argmax ( Vec f ) v → iargmax — greedy decoding is what whisper does by default, and what the reference implementations are compared at.
packages/whisper/src/ggml.nu — whisper.cpp's LEGACY ggml container.
The ggml-*.bin files ggerganov ships are how whisper models actually move between machines, and one is fully self-contained: hyperparameters, the tokenizer's vocabulary and every tensor in a single file — no config.json, no tokenizer.json. This reads that container so whisper transcribe ggml-tiny.bin clip.wav just works.
It is a PARSER, not a runtime: the model still runs on this package's own kernels. (The original goal said "not ggml like whisper.cpp" — that was about the ggml LIBRARY; the file format is just bytes, and hostile-input rules apply: every length is checked against the mapping before use.)
Layout (little-endian, no alignment padding):
u32 magic 'ggml' 0x67676d6c 11 × i32 hparams n_vocab, n_audio_ctx, n_audio_state, n_audio_head, n_audio_layer, n_text_ctx, n_text_state, n_text_head, n_text_layer, n_mels, ftype i32 n_mel, i32 n_fft, f32[n_mel*n_fft] mel filters (unused here — ours are verified against HF's) i32 n_vocab_in_file, then per word: i32 len, bytes (RAW bytes, not the HF byte-encoding) tensors until EOF: i32 n_dims, i32 name_len, i32 ttype, i32 dims[n_dims], name bytes, data
Two translations happen at load so the REST of the pipeline stays exactly the HF pipeline:
query.weight); the model layer asks in HF names (model.encoder.layers. 0.self_attn.q_proj.weight). gg_find translates.
bytes. The special tokens have no text at all — whisper.cpp derives their ids POSITIONALLY. We synthesize the HF spellings (<|startoftranscript|>, <|fi|>, <|0.00|> …) at those positions and byte-encode the base words, so name-based control-token lookup, the timestamp rules and tok_decode work unchanged.
: GgMeta: GgMeta {
i n_vocab
i n_audio_ctx
i n_audio_state
i n_audio_head
i n_audio_layer
i n_text_ctx
i n_text_state
i n_text_head
i n_text_layer
i n_mels
( Vec String ) vocab // the raw base words, in id order
( Vec String ) tnames // tensor names (ggml spelling)
( Vec i ) ttypes // 0 = f32, 1 = f16
( Vec i ) tnelems
( Vec i ) toffs // absolute byte offset of the tensor's data
}What the container says: the hyperparameters, the vocabulary and the tensor table — parsed once, never changed after.
: Gg: Gg {
( Arc GgMeta ) meta
( Locked ? MappedFile ) file // F once gg_release_data gave it back
}A Gg is a handle: every copy is the same container. The metadata is shared and frozen (an Arc); the file — an mmap where the platform has one (the page cache IS the copy, and a model that is already cached maps in with no I/O), the bytes read whole elsewhere — is a MappedFile the loader gives back once the weights are on the device: nothing reads the file after that, and 3 GB of a large model is not a thing to keep around for the vocabulary's sake. gg_close is an optional early release.
@ Gg_share Gg h → Gg@ Gg_drop sink Gg h → v@ gg_meta Gg h → GgMetaThe parsed container, lent.
@ gg_release_data Gg g → vGive the file back — the mapping or the read buffer — and keep the parsed metadata (vocabulary, tensor table).
@ gg_file_take Gg g → ?MappedFileThe mapped file, checked out for a reader that needs many views of it at once (a batched upload); every other use of g waits until gg_file_put returns it. F once gg_release_data gave it back.
@ gg_file_put Gg g sink ? MappedFile f → v@ gg_toff Gg h i ti → iWhere tensor ti's bytes start in the file; -1 for no such tensor.
@ gg_close sink Gg g → vEarly release (optional): the last owner gives everything back.
@ gg_open s path → !Gg StringOpen a container: mapped for the one sweep the loader makes over it (file_map_stream: every page mapped in one pass, read ahead in order).
@ gg_map_name s hf → StringTranslate an HF tensor name to whisper.cpp's OpenAI spelling.
@ gg_find Gg h s hf → iFind a tensor by its HF name. -1 when absent.
@ gg_ttype Gg h i ti → i@ gg_nelems Gg h i ti → i@ gg_build_tok Gg h → !Tok StringBuild the tokenizer: base words byte-encoded, specials synthesized at whisper.cpp's positional ids. Returns via tok_build so decode, control matching and the timestamp rules all behave exactly as with tokenizer.json.
packages/whisper/src/stream.nu — the live endpointer behind the WebSocket.
: ~ WsStream s ( ws_stream_new net cfg ) net: ?Silero (F = energy only) ( ws_stream_feed s samples ) 16 kHz audio as it arrives ~ T { : i act ( ws_stream_next s ) what is due, in stream order ? == act WSS_NONE { break } … START / PARTIAL / FINAL / LEVEL } ( ws_stream_end s ) the client says it is done
The stream decides WHEN; the server (serve.nu) does the transcribing. Each action names a stretch of the stream (act_from, act_to: absolute samples) and ws_stream_audio copies it out.
What is heard, 32 ms at a time:
detector, so a fan, music or a keyboard does not hold an utterance open, and a quiet word ending does not cut it short. With no weights it falls back to an energy margin over the noise floor.
(the 20th percentile of the last 10 s of non-speech chunks — the room, re-measured continuously, so a fan switched on is the floor within seconds), the SPEECH LEVEL (the median of the last ~6 s of voiced chunks — the speaker's own voice at their own microphone), and how twitchy Silero is on THIS room's noise (the 90th percentile of its p over chunks that are at the floor). Where the room makes Silero twitchy, its thresholds rise to stay above it.
near_db below the speaker'sown level is not the speaker — a television, a sibling across the room, the game's own sound. Without it, someone else talking is an utterance that never pauses.
When an utterance ends is the pause that follows it — and how long a pause has to be is the SPEAKER's, not a constant: the gaps between words inside an utterance are measured as they happen, and the silence that closes one is set from them (1.3 × their 90th percentile + 150 ms, within [min_pause_ms, max_pause_ms]). A fluent speaker's 120 ms gaps close an answer after ~300 ms; a child reading word by word with 600 ms gaps is not cut after every word. A close followed quickly by more speech counts as a gap as well, so a threshold that keeps splitting a speaker's sentences grows until it stops.
Drafts (cfg.partials): the open utterance is transcribed every partial_ms of new speech, and once more as soon as a pause starts — that one with the full encoder context, so when the pause turns out to be the end, the draft IS the final and nothing has to run.
: i WSS_NONE 0: i WSS_START 1 // speech confirmed at act_from: i WSS_PARTIAL 2 // a draft of [act_from, act_to) is due (act_full: the full context): i WSS_FINAL 3 // the utterance [act_from, act_to) closed: i WSS_LEVEL 4 // a second went by: the levels, for a client that asked: i __WSS_RATE 16000: i __WSS_HOP 512 // one Silero chunk, 32 ms: WssCfg: WssCfg {
b partials // send drafts
i partial_ms // a draft every this much new audio
i pause_ms // the closing silence before any gap has been measured
i min_pause_ms
i max_pause_ms
b adaptive // F = pause_ms, always
f threshold // Silero p that opens an utterance (it stays open down to −0.15)
f near_db // the near-field gate (0 = off)
i max_utt_ms // an utterance this long is closed at its best recent gap
b levels // WSS_LEVEL once a second
b events // WSS_START
}A stream's tunables — the client may set any of them in its config message (serve.nu), at any time.
@ wss_default_cfg → WssCfg: WsStream: WsStream {
WssCfg cfg
? Silero net
SileroState sst
( Vec f ) buf // samples not yet released; buf[0] is absolute sample `base`
i base
i pos // the endpointer has heard everything before this sample
// the trackers (rings of chunk levels in dB; p of the noise chunks)
( Vec f ) noise_e
( Vec f ) noise_p
i noise_i
i noise_pi
( Vec f ) all_e
i all_i
( Vec f ) voice_e
i voice_i
( Vec f ) pw3 // the last three chunks' power: the level the gates read
f noise_db
f speech_db
f th_on
f th_off
f p_noise // Silero's 90th-percentile p on this room's noise
i nchunk
// the open utterance
i seg // utterances started so far
i seg_start // absolute sample (−1 = none open)
i onset // where its speech began
i last_voice // end of the last voiced chunk
i voiced_n
b confirmed
i split_at // the middle of the latest real gap (a force-close point)
// pauses
( Vec i ) gaps // the last 16 gaps between words, ms
i gaps_i
i eou_ms // the silence that closes an utterance now
i last_end // where the last utterance's speech ended (absolute), −1 = none
// drafts
i draft_at // the stream position the last draft heard up to (−1 = none)
i draft_voice // last_voice when it was made
i full_voice // … and when the last full-context one was
b full_ok // … and that one did run (a skipped one is not retried)
b draft_full
// the action being handed out
i act_seg // the utterance it belongs to (1, 2, …)
b start_due
b final_due
i act_from
i act_to
b act_full
b act_reuse // FINAL: the last draft heard exactly this — no need to run again
i act_pause_ms // how long the speaker had been quiet at act_to
i act_speech_from // the speech itself: onset …
i act_speech_to // … to the end of the last voiced chunk
i level_at
}@ ws_stream_new ? Silero net WssCfg cfg → WsStream@ ws_stream_set_cfg inout WsStream s WssCfg cfg → vNew settings for a running stream. A pause or threshold the client sets takes effect at once; the measured gaps are kept.
@ ws_stream_feed inout WsStream s ( Vec f ) x → v@ ws_stream_audio WsStream s i from i to → ( Vec f )The samples [from, to) of the stream (what is still held of them).
@ ws_stream_next inout WsStream s → iWhat is due next. Runs the endpointer forward chunk by chunk and stops at the first action, so each one is handed out at the moment in the stream it became due (a draft hears exactly the audio up to its act_to).
@ ws_stream_draft_skipped inout WsStream s → vThe draft just handed out did not run (the GPU was busy, or the stream's share of it was spent): a full-context one can no longer stand in for the final.
@ ws_stream_end inout WsStream s → bThe client is done (or gone): whatever is open closes now, over all the audio received. T = a FINAL is due.
@ ws_stream_reset inout WsStream s → vDrop the open utterance unheard (a new question: what was being said is no longer an answer). The room's measurements are kept.
@ wss_words ( Vec u ) t → ( Vec i )Word boundaries: (start, end) byte pairs.
@ wss_agree ( Vec u ) a ( Vec u ) b → iHow many leading words two transcripts share.
@ wss_prefix ( Vec u ) t i n → ( Vec u )The first n words of t, as it spelt them (leading space included, the way whisper starts a transcript).
packages/whisper/src/run.nu — the shared middle of the pipeline: what transcribe and serve both do once a model and a tokenizer are open.
whis_ggml / whpath which container a model argument names wh_run 16 kHz mono float samples → the words (with --vad's condensed timeline mapped back, and whisper's own timestamp tokens when asked)
The CLI (main.nu) and the server (serve.nu) import this rather than each other: every source module compiles on its own, which is what the publish gate checks.
@ _wh_is_ggml s model → bIs model a whisper.cpp ggml container? Decided by the file's own first bytes ('lmgg' on disk), not the extension — a renamed file still works and a directory never opens as a file.
@ _wh_path s dir s name → String<dir>/<name>
@ wh_wav16 sink Wav aw → ( Vec f )Audio longer than 30 seconds is transcribed in 30-second WINDOWS: the encoder sees exactly 30 s (that is the length it was trained on and the length its positional embedding has), so a longer clip is split, and each window is decoded from a fresh prompt with its own KV cache. The transcription core, from samples to text: VAD (optional), the 30-second window loop, decode. TAKES OWNERSHIP of at16_in (16 kHz mono) — the VAD path replaces the buffer wholesale, so the caller's handle is dead either way, and this function frees whichever buffer survives.
This is the seam the server stands on: the CLI opens the model, runs this once and exits; whisper serve opens the model ONCE and runs this per request — the 1.5 GB read, the f16→f32 conversion and the kernel compile all happen before the first request instead of inside every one. A WAV as the 16 kHz mono samples the model hears. Takes the WAV: it and its mono mix are gone when this returns, so a long recording is not held three times over while the model runs (an hour at 16 kHz is 460 MB a copy).
@ wh_run Whisper w Tok t sink ( Vec f ) at16_in s lang i maxtok b use_vad b with_ts f nospeech ( Vec u ) out → b@ wh_run_seg Whisper w Tok t ( Vec f ) seg i ctx s lang i maxtok f nospeech ( Vec u ) out → bOne utterance of at most 30 s, as plain text, for a stream: no VAD (the stream has already segmented), no timestamps (the segment is the timestamp). ctx as in __wh_window; 0 = the full window.
packages/whisper/src/silero.nu — Silero VAD v5 (the 16 kHz network) on the CPU.
( silero_open path ) → !Silero String ( silero_state_new ) → SileroState one per stream ( silero_prob net st x off ) → f p(speech) of x[off, off+512)
An energy detector hears "loud enough"; this hears "a voice". A fan, a game's music, a keyboard or a door is loud and is not speech, and a quiet "neljä" at the end of a sentence is speech and is not loud — the cases a margin over a noise floor gets wrong in both directions. faster-whisper and whisper.cpp both ship Silero for that reason, and it is small enough to run on the CPU between WebSocket frames: 309 k weights, ~0.7 M multiply-adds per 32 ms of audio.
The network, chunk by chunk (512 samples = 32 ms at 16 kHz): 64 samples of the previous chunk + the chunk, reflect-padded by 64 on the right → a 256-point STFT, hop 128 (4 frames × 129 magnitudes) → four conv1d (k3, pad 1; strides 1 2 2 1) with ReLU: 129→128→64→64→128 → one LSTM cell step (128), its state carried to the next chunk → ReLU, a 1×1 conv to one logit, sigmoid. Checked against onnxruntime on the original ONNX file: max |Δp| 2·10⁻⁶.
The weights are vad/silero_vad_v5.safetensors (tests/silero_export.py takes them from snakers4/silero-vad's ONNX file; MIT, vad/LICENSE-silero-vad). The STFT basis is not in the file: it is a periodic-Hann DFT, computed here.
: i SILERO_CHUNK 512The chunk the network was trained on, and the context it carries over.
: i __SV_CTX 64: SileroW: SileroW {
( Vec f ) cosb // [129][256]: periodic Hann × cos
( Vec f ) sinb // [129][256]: periodic Hann × −sin
( Vec f ) w0 // [128][129][3]
( Vec f ) b0
( Vec f ) w1 // [64][128][3], stride 2
( Vec f ) b1
( Vec f ) w2 // [64][64][3], stride 2
( Vec f ) b2
( Vec f ) w3 // [128][64][3]
( Vec f ) b3
( Vec f ) wih // [512][128], gates i f g o
( Vec f ) whh // [512][128]
( Vec f ) bsum // bias_ih + bias_hh
( Vec f ) wd // [128]
f bd
}: Silero { ( Arc SileroW ) a }The network, read-only once loaded: every copy is the same weights, and every stream that runs it keeps its own SileroState.
@ Silero_share Silero h → Silero: SileroState: SileroState {
( Vec f ) h
( Vec f ) c
( Vec f ) ctx
}The recurrent state and the 64-sample context of one stream.
@ silero_state_new → SileroState@ silero_reset inout SileroState st → vForget everything heard: the state a fresh stream starts in.
@ silero_open s path → !Silero StringLoad the network from a safetensors file holding the 16 kHz branch's weights under their PyTorch names (what tests/silero_export.py writes).
@ silero_default_path → StringWhere the weights live when no path is given — the same places the test page is looked for (nurlpkg stages [install].assets into share/whisper/): <exe-dir>/vad/…, <exe-dir>/../share/whisper/vad/…, ./vad/… (package dir). "" = not found.
@ silero_prob Silero net inout SileroState st ( Vec f ) x i off → fp(speech) for the 512 samples x[off, off+512), advancing st past them. Chunks have to be fed in order and without gaps: the LSTM state and the 64-sample context are what the previous chunk left.