NURLNURL registrynurl-lang.org →

← f5tts

f5tts 0.3.0 API

registry.nu

packages/f5tts/src/registry.nu — which models this machine can speak with.

There is no built-in catalogue. This package names no checkpoint, ships no list of "known" models and has no default one, because a speech model is a choice about a language, a voice and a licence, and none of those are this program's to make. A model is named in one of three ways and they are tried in that order:

A PATH an existing file or a directory holding a checkpoint and a vocab.txt. Whatever is on the disk.

A LOCAL ID the name of a directory under ~/.f5tts/models. This is where a fine-tune lands, or a training run's output, or a file copied off another machine. GET /models lists these.

A REFERENCE owner/repo/path-to-file, which the hub fetches into the shared ~/.nurl/models cache the first time and costs nothing after that. The vocabulary is taken to be vocab.txt beside the checkpoint in the same repository, which is how every F5-TTS release is laid out; name it with --vocab when it is not.

A fresh machine has no local models and can reach any reference, which is the state this is designed around: f5tts serve --model <reference> on a machine that has just installed it works, and the first request pays the download.

API

: F5Entry

: F5Entry {
    String id
    String ckpt  // a local path, or a Hugging Face reference
    String vocab  // the same
    b local  // a directory under the models dir rather than a reference
    b cached  // the files are on this machine already
}

@ f5_is_reference s id → b

An owner/repo/file reference rather than a plain name: a separator makes it one, and f5_id_ok has already refused the separator to a local id, so the two can never be confused.

@ f5_vocab_beside s ckpt_ref → String

vocab.txt beside the checkpoint, in whatever names the checkpoint — a repository reference or a directory on this machine.

@ f5_registry s models_dir → ( Vec F5Entry )

Everything this machine has on its own disk. A reference the machine has fetched is not here — it belongs to whoever asked for it, by name.

@ f5_registry_has ( Vec F5Entry ) v s id → b

@ f5_registry_resolve s models_dir s id String ckpt_out String vocab_out → b

Resolve a name to the two paths a model needs, fetching if the name is a repository reference this machine has not pulled yet. False = no such model, or the fetch failed (which is reported on stderr).

vocos.nu

packages/f5tts/src/vocos.nu — the vocoder: a mel spectrogram back to sound.

F5-TTS produces a mel spectrogram, which is not audio. Vocos is what makes it audible, and it is a refreshingly honest piece of engineering: no transposed convolutions, no upsampling stack, no adversarial magic at inference time. It runs eight ConvNeXt blocks at the SPECTROGRAM's frame rate — one frame stays one frame all the way through — and then predicts, for each frame and each of the 513 frequency bins, a log-magnitude and a phase. An inverse STFT turns those into samples.

So the upsampling from 24000/256 frames per second to 24000 samples per second happens once, in the overlap-add, and it is arithmetic rather than a learned layer. That is why the whole vocoder is 13.6 million parameters where a HiFi-GAN is ten times that.

The checkpoint is a PyTorch .bin — a zip of a pickle — so it is read through packages/torchpt rather than safetensors.

API

: VocosImpl

: VocosImpl {
    GpuKit kit
    Pt pt  // the mmapped .bin
    i dim
    i inner
    i layers
    i nmel
    i nfft
    i hop
    GkBuf emb_w GkBuf emb_b
    GkBuf n_w GkBuf n_b
    ( Vec GkBuf ) dw_w ( Vec GkBuf ) dw_b
    ( Vec GkBuf ) nw ( Vec GkBuf ) nb
    ( Vec GkBuf ) p1w ( Vec GkBuf ) p1b
    ( Vec GkBuf ) p2w ( Vec GkBuf ) p2b
    ( Vec GkBuf ) gam
    GkBuf fn_w GkBuf fn_b
    GkBuf out_w GkBuf out_b
    // scratch, sized by the frame count
    i frames
    GkBuf melb GkBuf h GkBuf t1 GkBuf t2 GkBuf big GkBuf spec
}

: Vocos { s ctl }

A Vocos is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same vocoder, and the last owner releases it.

@ Vocos_share Vocos h → Vocos

@ Vocos_drop sink Vocos h → v

@ voc_open s path GpuKit kit → !Vocos String

@ voc_close sink Vocos v → v

Early release (optional): the last owner of the vocoder gives back its weights, its scratch and the mapping.

@ voc_decode Vocos v__h ( Vec f ) mel i frames ( Vec f ) out → b

mel (frames × 100, row-major, natural-log scale) → waveform at 24 kHz.

@ voc_loaded Vocos v__h → b

@ voc_unload Vocos v__h → v

@ voc_reload Vocos v__h → b

model.nu

packages/f5tts/src/model.nu — the DiT that F5-TTS calls a transformer, and the weights it reads.

It is a diffusion transformer, not a language model, and almost everything that follows is a consequence of that. There is no KV cache and no causal mask: every step of the ODE looks at the whole utterance at once, and the same 22 blocks run again from scratch for every one of them. What changes between steps is a single scalar — the time — and it reaches the network not as a token but as a per-feature SCALE and SHIFT applied to every normalised activation, plus a GATE on every residual. That is adaLN-zero, and it is why a DiT block has a 6·dim projection hanging off the timestep that a llama block has no equivalent of.

The text path is stranger still. The characters are not a sequence the audio attends to; they are UPSAMPLED to the audio's own length — one character per mel frame, zero-padded to the end — embedded, run through four ConvNeXt-V2 blocks, and CONCATENATED onto the noised mel frame by frame. So the model never learns an alignment; the duration estimate chooses one, and the text is stretched to fit it.

Layout: every activation is row-major [batch·n, features], batch-major, so a two-sample classifier-free-guidance forward is one tall GEMM rather than two. The attention needs [batch·heads, n, hd], and the kernel that splits the heads applies the rotation on the way through.

API

: f F5_PI 3.14159265358979323846

: F5Scratch

: F5Scratch {
    GkBuf xbuf  // [n, mel]        the ODE state, shared by both CFG rows
    GkBuf cond2  // [batch*n, mel]  the masked reference mel, zeroed for uncond
    GkBuf txt2  // [batch*n, td]   the text encoder's output, cond then uncond
    GkBuf cat  // [batch*n, 2*mel+td]
    GkBuf h  // [batch*n, dim]  the residual stream
    GkBuf hn  // [batch*n, dim]  the normalised copy
    GkBuf tmp  // [batch*n, dim]
    GkBuf qb GkBuf kb GkBuf vb GkBuf ob
    GkBuf qkv  // [batch*n, 3*dim] — the fused projection's output
    GkBuf ff  // [batch*n, ffmult*dim]
    GkBuf pred  // [batch*n, mel]
    GkBuf vel  // [n, mel]
    GkBuf temb  // [dim]
    GkBuf tsin  // [freq_embed_dim] the sinusoidal timestep features
    GkBuf mod6  // [6*dim]
    GkBuf mod2  // [2*dim]
    GkBuf cosd GkBuf sind  // [n, hd/2]
    GkBuf tscr  // [n, 2*td] the ConvNeXt block's wide intermediate
    GkBuf tscr2
    GkBuf tscr3
    GkBuf tsil  // [dim] SiLU(timestep embedding), shared by all 23 modulations
    GkBuf grn_gx GkBuf grn_mean
    GkBuf keep  // [n] 1.0 where the text has a character
}

The per-generation buffers (n is the mel length), as one value: f5_alloc replaces the whole set, and the old one is dropped as a unit.

: F5ModelImpl

: F5ModelImpl {
    GpuKit kit
    St st  // a safetensors checkpoint, st_none when this is a .pt
    Pt pt  // a PyTorch pickle checkpoint, pt_none when this is safetensors
    String prefix  // what every tensor name in this file starts with
    // architecture
    i dim
    i depth
    i heads
    i hd
    i ffmult
    i td  // text_dim
    i mel
    i nconv  // ConvNeXt blocks in the text encoder
    i cp_k  // the position embedding's kernel width
    i cp_groups
    i tb_k  // the text encoder's depthwise kernel width
    i vocab
    // timestep embedding
    GkBuf tm0_w GkBuf tm0_b GkBuf tm2_w GkBuf tm2_b
    // text embedding
    GkBuf temb_w
    ( Vec GkBuf ) tb_dw_w ( Vec GkBuf ) tb_dw_b
    ( Vec GkBuf ) tb_n_w ( Vec GkBuf ) tb_n_b
    ( Vec GkBuf ) tb_p1_w ( Vec GkBuf ) tb_p1_b
    ( Vec GkBuf ) tb_gg ( Vec GkBuf ) tb_gb
    ( Vec GkBuf ) tb_p2_w ( Vec GkBuf ) tb_p2_b
    // input embedding + convolutional position embedding
    GkBuf ie_w GkBuf ie_b
    GkBuf cp0_w GkBuf cp0_b GkBuf cp2_w GkBuf cp2_b
    // transformer blocks
    GkBuf an_all  // the 22 timestep projections, stacked into one matrix
    GkBuf anb_all
    // the modulation for EVERY step, computed before the loop starts
    GkBuf mod_all
    GkBuf mod2_all
    i nsteps
    i cur_step
    ( Vec GkBuf ) wqkv ( Vec GkBuf ) bqkv  // q, k and v stacked: one GEMM, not three
    ( Vec GkBuf ) wq ( Vec GkBuf ) bq
    ( Vec GkBuf ) wk ( Vec GkBuf ) bk
    ( Vec GkBuf ) wv ( Vec GkBuf ) bv
    ( Vec GkBuf ) wo ( Vec GkBuf ) bo
    ( Vec GkBuf ) f1_w ( Vec GkBuf ) f1_b
    ( Vec GkBuf ) f2_w ( Vec GkBuf ) f2_b
    // output
    GkBuf no_w GkBuf no_b GkBuf po_w GkBuf po_b
    // ── per-generation scratch (n is the mel length) ──
    i n
    i batch
    F5Scratch sc  // the per-generation buffers (f5_alloc)
    b ready
    b loaded  // the weights are on the device
}

: F5Model { s ctl }

An F5Model is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same engine, and the last owner releases it.

@ F5Model_share F5Model h → F5Model

@ F5Model_drop sink F5Model h → v

@ f5_open s ckpt s vocab_path i device → !F5Model String

A checkpoint is a path to a .safetensors or a .pt. Which container, which prefix and which architecture are all read out of the file: nothing here is told what it is about to open.

@ f5_close sink F5Model m → v

Free every buffer in the list but KEEP the list — an unload empties it and a reload fills it again. Early release (optional): the last owner of the engine gives back the weights, the scratch, the mapping and the kit.

@ f5_vocab_n F5Model m__h → i

@ f5_alloc F5Model m__h i n i batch → b

@ f5_text_encode F5Model m__h ( Vec i ) ids b drop i row_off → b

The text encoder's whole output for one classifier-free-guidance row, written into txt2 at row_off. drop is the unconditional branch: the characters become the filler token, but the PADDING MASK is still the real text's — which is why the unconditional text embedding is not zero.

@ f5_set_times F5Model m__h ( Vec f ) ts → b

── the timestep ────────────────────────────────────────────────────

F5-TTS's timestep features are sin FIRST then cos, over exp(-i·ln(10000)/127) scaled by a thousand — a different ladder and a different order from the position code its text encoder uses. Getting this backwards does not crash; it just conditions the whole network on the wrong time. Every step's conditioning, in one pass.

The timesteps are a schedule, not a discovery: they are known before the first forward. So the sinusoidal features, the two-layer MLP, the SiLU and all 23 modulation projections are computed for all of them at once — which turns 22 matrix-VECTOR products per step into one matrix-matrix product per utterance, and reads the 554 MB of modulation weights once instead of once per step.

F5-TTS's timestep features are sin FIRST then cos, over exp(-i·ln(10000)/127) scaled by a thousand — a different ladder and a different order from the position code its text encoder uses. Getting this backwards does not crash; it conditions the whole network on the wrong time.

@ f5_set_step F5Model m__h i k → v

@ f5_forward F5Model m__h → b

@ f5_rope F5Model m__h → b

@ f5_buf_txt F5Model m__h → GkBuf

@ f5_buf_pred F5Model m__h → GkBuf

@ f5_buf_x F5Model m__h → GkBuf

@ f5_buf_cond F5Model m__h → GkBuf

@ f5_kit F5Model m__h → GpuKit

@ f5_n F5Model m__h → i

@ f5_mel F5Model m__h → i

@ f5_download F5Model m__h GkBuf b ( Vec f ) out i nel → b

Read nel elements of a device buffer back, growing out to fit.

@ f5_set_x F5Model m__h ( Vec f ) x → b

@ f5_set_cond F5Model m__h ( Vec f ) c → b

@ f5_zero_cond_row F5Model m__h i row_off → b

The unconditional half's conditioning audio is zero: classifier-free guidance drops the reference voice, not only the text.

@ f5_buf_vel F5Model m__h → GkBuf

@ f5_loaded F5Model m__h → b

@ f5_unload F5Model m__h → v

@ f5_reload F5Model m__h → b

@ f5_set_one_time F5Model m__h f t → b

One timestep, for a caller that has exactly one — the tensor-by-tensor tests, which compare a single forward against the reference.

@ f5_dim F5Model m__h → i

@ f5_depth F5Model m__h → i

@ f5_heads F5Model m__h → i

@ f5_td F5Model m__h → i

@ f5_prefix F5Model m__h → s

serve.nu

packages/f5tts/src/serve.nu — the synthesis service.

Shaped like the deployed FastAPI one: POST /tts and POST /dialogue take the same fields and answer with a wav, GET /voices lists what can speak, GET /health says whether the weights are on the card.

One thread owns the GPU. Connections are fibers — cheap, and they spend their time on sockets and JSON — but the forward is handed to ONE dedicated model thread over a queue, one job per request. Three reasons, and the third is the one that bites:

the same one;

would be two forwards through the same scratch;

So a dialogue of eight lines is eight jobs, each answered in turn, and a second client's request queues behind the first rather than corrupting it.

--unload-after N gives the card back. The model thread is the only one that touches the device, so it is the one that lets go: a wake with nothing queued and the idle clock past the limit unloads, and the next job reloads before it runs. The wakes come from a ticker thread, because a thread asleep on a condition variable has no way to notice time passing.

API

: ~ i g_f5_model 0 // an F5Model's ctl word: the global is one of its owners

: ~ i g_f5_voc 0 // a Vocos's ctl word, likewise

: ~ i g_f5_vocab 0 // an F5Vocab's ctl word, likewise

: ~ s g_f5_voices // the voices directory

: ~ s g_f5_token ``

: ~ s g_f5_vocoder ``

: ~ i g_f5_reqs 0

: ~ s g_f5_models_dir ``

: ~ i g_f5_cur_id 0 // String — the model the thread currently holds

: ~ i g_f5_switches 0

: ~ i g_f5_unload_ms 0

: ~ i g_f5_idle_since 0

: ~ i g_f5_loads 1

: ~ i g_f5_unloads 0

: ~ i g_f5_load_ms 0

: F5JobImpl

: F5JobImpl {
    i next
    String voice
    String text
    i steps
    f cfg
    f sway
    f speed
    f fade
    i seed
    i retries
    f max_wer
    i splitfail
    f target_rms
    String model_id
    ( Vec f ) out
    ( Vec i ) score
    b done
    b ok
    String err
}

: F5Job { s ctl }

@ F5Job_drop sink F5Job h → v

: ~ i g_q_head 0

: ~ i g_q_tail 0

: ~ b g_q_stop F

: F5Sync

: F5Sync {
    Mutex m  // guards the queue, g_q_stop and the idle clock
    Cond req  // a job was queued, the server is stopping, or a ticker wake
    Cond done  // a job was finished
}

: ~ i g_q_sync 0 // *F5Sync as an address (0 = never served)

: ~ i g_vc_ids 0 // ( Vec String ) as a raw handle

: ~ i g_vc_ptrs 0 // ( Vec F5Voice ) as a raw handle — the cache owns them

: ~ i g_vc_rms 0 // ( Vec f ) — the loudness each was prepared at

@ f5_voice_id_ok s id → b

A voice id has to be a plain name: no separator, no dots. The directory is joined onto a path the server owns, and a request does not get to walk out of it.

@ f5_serve s ckpt s vocab_path s vocoder s voices_dir s models_dir s model_id

verify.nu

packages/f5tts/src/verify.nu — listening to what came out.

A flow-matching model given a duration estimate and some noise does not always say the words. It drops a short one, it runs two together, it mumbles the end of a chunk whose duration guess was tight. There is no signal inside the model that says so — the velocity field is just as confident either way.

So the deployed service checks by LISTENING: it transcribes what it generated, compares that against the text it was asked for, and if the word error rate is over a threshold it generates the segment again from a different seed. That loop is the difference between a demo and a service, and it is why the Python one is configured with WHISPER_HOST.

This talks to the same endpoint — POST /inference with a wav, {"text": …} back — which whisper.cpp's server and packages/whisper's both speak.

API

: ~ s g_f5v_host ``

: ~ i g_f5v_port 6543

: ~ s g_f5v_lang // empty = let the transcriber detect one

@ f5_whisper_set s host i port s lang → v

The transcriber is process configuration: set once at startup, so the strings are copied into the globals and kept for the process's life.

@ f5_whisper_enabled → b

@ f5_whisper_where → String

@ f5_number_words i n → String

@ f5_words s text → ( Vec String )

The words of a normalised text.

: i F5V_MERGE 4

@ f5_word_errors ( Vec String ) r ( Vec String ) h → i

Edit distance over WORDS with one extension: a word on either side may match the concatenation of up to F5V_MERGE consecutive words on the other at no cost. "lepakon kosto" against "lepakonkosto" is then zero errors, as it should be — the speaker said both words, the transcriber wrote one. Returns the distance; the caller divides by the reference's length.

@ f5_errors s reference s hypothesis → i

The transcriber's word errors against the text, and the text's own word count as the gate counts it — normalised, digits spelt out.

@ f5_word_count_norm s text → i

@ f5_wer s reference s hypothesis → f

The word error rate: errors over the reference's word count.

@ f5_transcribe ( Vec f ) wave → String

kernels.nu

packages/f5tts/src/kernels.nu — the kernels a flow-matching DiT needs and a language model does not.

The heavy arithmetic — every projection, the feed-forward, the fused attention — is gpukit's: its GEMM is tiled and tuned, and its attention is the online-softmax one that never writes a score matrix. What is here is what F5-TTS has and the rest of this ecosystem does not:

vector; it derives a scale and a shift per feature from the timestep and applies them to a normalised activation, and it gates the residual by a third one. Fused into the LayerNorm, that is one pass over the activation instead of four.

[n, heads*hd]; attention wants [heads, n, hd]; the rotation is a per-pair 2-D rotation. All three are one kernel, so the activation is read once and written once.

encoder's ConvNeXt blocks, and GROUPED with kernel 31 for the convolutional position embedding. Neither exists anywhere else here.

down the SEQUENCE rather than across the features — two reductions over the whole block before a single elementwise pass.

purpose: the text encoder uses the exact error-function form and the transformer's feed-forward uses the tanh approximation, and they are different functions.

Every kernel is written once and runs on both of gpukit's backends: the CPU one implements shared as thread-local storage and __syncthreads as a fiber yield, so a block-wide reduction is portable.

API

@ f5k_modln GpuKit kit i xd i yd i scaled i shiftd i rows i d f eps → b

@ f5k_lnaff GpuKit kit i xd i yd i wd i bd i rows i d f eps → b

@ f5k_gated_add GpuKit kit i xd i yd i gated i rows i d → b

@ f5k_split_rope GpuKit kit i srcd i dstd i cosd i sind i batch i n i heads i hd i dorope → b

@ f5k_split_rope_s GpuKit kit i srcd i dstd i cosd i sind i batch i n i heads i hd

@ f5k_merge GpuKit kit i srcd i dstd i batch i n i heads i hd → b

@ f5k_gelu_erf GpuKit kit i xd i n → b

@ f5k_gelu_tanh GpuKit kit i xd i n → b

@ f5k_silu GpuKit kit i xd i n → b

@ f5k_mish GpuKit kit i xd i n → b

@ f5k_conv1d GpuKit kit i xd i yd i wd i bd i batch i n i cin i cout i K i pad i groups → b

One grouped convolution covers all three shapes this port needs: depthwise (groups = channels), fully connected across channels (groups = 1) and the position embedding's sixteen groups.

@ f5k_conv1d_t4 GpuKit kit i xd i yd i wd i bd i batch i n i cin i cout i K i pad i groups → b

The transposed-weight form. Same arguments, same result. The four-position form. Same weights, same result, a quarter of the weight traffic.

@ f5k_conv1d_t GpuKit kit i xd i yd i wd i bd i batch i n i cin i cout i K i pad i groups → b

@ f5k_grn GpuKit kit i xd i gxd i meand i gammad i betad i rows i d → b

@ f5k_maskrows GpuKit kit i xd i keepd i rows i d → b

@ f5k_concat3 GpuKit kit i xd i condd i txtd i outd i batch i n i mel i td → b

@ f5k_cfg GpuKit kit i predd i outd i n i d f cfg → b

@ f5k_axpy GpuKit kit i yd i vd f a i n → b

@ f5k_addinto GpuKit kit i yd i ad i n → b

@ f5k_scalecols GpuKit kit i xd i gd i rows i d → b

@ f5k_dup GpuKit kit i srcd i dstd i n i copies → b

text.nu

packages/f5tts/src/text.nu — the vocabulary, and the exact character sequence an F5-TTS checkpoint was trained to read.

F5-TTS calls its text front-end convert_char_to_pinyin, and for Chinese that is what it does. For a Latin-script language it does something else entirely, and the something else is not cosmetic. The text is segmented by jieba first, and jieba's "han" character class — [一-鿕a-zA-Z0-9+#& ._%-] — does NOT contain ä, ö or å. A Finnish word therefore breaks into pieces at every umlaut, and the rule that puts a space in front of a multi-character ASCII segment then fires INSIDE the word:

"Yöllä" segments: Y | ö | ll | ä characters: Y ö _ l l ä

A checkpoint for such a language was fine-tuned through this same front-end, so the inserted space is what the model expects to read. Reproducing it is not bug-compatibility for its own sake: drop it and every umlauted word becomes a word the model has never seen.

Everything jieba does to a Latin run beyond that is nothing — a run of han-class bytes comes back out whole (the DAG finds no dictionary word in it, the buffer flush hands it to finalseg, and finalseg's own skip class yields alphanumerics unsplit). So the segmentation needs no dictionary and no Viterbi: it is a three-way byte classification, done here in one pass.

( f5_vocab_load path ) → !F5Vocab String (a handle; the last owner releases it) ( f5_text_ids vocab text out ) → v ids appended to out ( f5_chunk_text text max_bytes ) → ( Vec String )

API

: F5VocabImpl

: F5VocabImpl {
    ( Vec i ) ascii  // 128 slots: a one-byte character's id, -1 when absent
    ( HashMap s i ) map  // a multi-byte character → its id; keys point into `keys`
    ( Vec u ) keys  // every multi-byte character, NUL-terminated, back to back
    i size  // the vocabulary's length, which is also text_num_embeds
}

: F5Vocab { s ctl }

An F5Vocab is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same vocabulary, and the last owner releases it.

@ F5Vocab_share F5Vocab h → F5Vocab

@ F5Vocab_drop sink F5Vocab h → v

@ _f5t_geti ( Vec i ) v i k → i

The same read, public: run.nu's silence walk needs it too.

@ f5_vocab_load s path → !F5Vocab String

Read a vocab.txt — one token per line, the line NUMBER is the id. Python reads it as for i, char in enumerate(f) and drops the last character of each line, so a trailing "\r" from a CRLF file would be part of the token; a file ending in a newline yields no extra empty token.

@ f5_vocab_size F5Vocab v__h → i

@ f5_vocab_free sink F5Vocab v → v

Early release (optional): the last owner of a vocabulary releases it.

@ f5_text_ids F5Vocab v__h s text ( Vec i ) out → v

The character sequence, as vocabulary ids, appended to out.

@ f5_chunk_text s text i max_bytes → ( Vec String )

Split into sentences the way the regex does, then pack.

ui.nu

packages/f5tts/src/ui.nu — the page the server answers GET / with.

One file, no build step, no CDN: the markup, the style and the script are a string in the binary, so a machine that has just installed f5tts and has no network still gets a working interface. It talks to the same endpoints as any other client — /voices, /models, /tts, /dialogue, /voices/add — so anything visible here is reachable with curl, and anything that works with curl shows up here.

Nothing here names a model or a language. The model list is whatever the machine has, the voices are whatever somebody recorded, and the text box starts empty: a prefilled sentence in one language is a default for that language and a nuisance in every other.

API

@ f5_ui_html → s

run.nu

packages/f5tts/src/run.nu — a recording, a transcript and a sentence in; a wav out.

The model is only the middle of this. Around it sits a set of rules that are not learned and are not documented anywhere except in the reference implementation's control flow, and every one of them changes what comes out:

assumes the new text will take as long per byte as the reference did:

frames = ref_frames + ref_frames/ref_bytes · gen_bytes / speed

That estimate IS the alignment — the text is stretched to exactly that many frames — so it is also the single biggest lever on how the result sounds. Under ten bytes of text the estimate is scaled by 0.3, because a very short utterance given its linear share of the time comes out clipped before the model has settled.

result is scaled back by the same factor. The model hears a constant loudness whatever the recording was; the caller gets their own back.

reference's own speaking rate and how much of a 22-second window it already occupies, and the pieces are CROSS-FADED over 150 ms rather than butt-joined.

API

: i F5_SR 24000

: i F5_HOP 256

: f F5_TARGET_RMS 0.1

: F5VoiceImpl

: F5VoiceImpl {
    ( Vec f ) mel  // frames × 100
    i frames
    i samples  // of the normalised recording, which sets ref_audio_len
    f rms  // the recording's own loudness, to scale the result back
    f target  // the loudness it was normalised TO, so a cache can tell
    String text
}

A voice: the reference recording, already at 24 kHz mono and loudness normalised, plus its mel and its transcript.

: F5Voice { s ctl }

An F5Voice is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same voice, and the last owner releases it.

@ F5Voice_share F5Voice h → F5Voice

@ F5Voice_drop sink F5Voice h → v

@ f5_voice_target F5Voice v__h → f

@ f5_voice_free sink F5Voice v → v

Early release (optional): the last owner of a voice releases it.

@ f5_fix_ref_text s t → String

F5-TTS insists the reference transcript ends in a sentence break — without one the model runs the reference and the new text together as one breath.

@ f5_voice_load s wav_path s ref_text f target_rms → !F5Voice String

@ f5_voice_open_dir s dir f target_rms → !F5Voice String

The voice directory the deployed service uses: config.json holds the transcript, reference.wav the recording.

@ f5_max_chars F5Voice v__h f speed → i

── the byte budget for one chunk ───────────────────────────────────

How much text fits beside a reference of this length inside the 22-second window the model was trained on, at the reference's own speaking rate.

: ~ b g_f5r_shortfix F

The frame count F5-TTS will generate for one chunk. gen_bytes is the new text's length in BYTES, not characters — a Finnish umlaut counts twice, and that is the reference implementation's own arithmetic, not an oversight this port kept. Off by default: the reference implementation's rule is one line, and a port that quietly used a different one would not be a port. What turning it on does, and why it exists, is in the comment inside f5_duration.

@ f5_short_fix b on → v

@ f5_short_fix_on → b

: i F5_SHORT_PLUS_MS 1100

: i F5_SHORT_FULL_BYTES 30

: i F5_SHORT_TAPER_END 120

@ f5_duration F5Voice v__h s gen_text i n_text f speed → i

@ f5_ref_audio_len F5Voice v__h → i

@ f5_score_new → ( Vec i )

@ f5_score_errs ( Vec i ) s → i

@ f5_score_words ( Vec i ) s → i

@ f5_score_attempts ( Vec i ) s → i

@ f5_score_unheard ( Vec i ) s → i

@ f5_score_checked ( Vec i ) s → b

@ f5_score_wer ( Vec i ) s → f

The word error rate the score stands for, or -1 when nothing was checked.

@ f5_score_merge ( Vec i ) into ( Vec i ) from → v

@ f5_gate_on i retries f max_wer → b

Is the gate on at all for these settings? A transcriber, and something that could act on its answer.

: i F5_LISTEN_TRIES 4

The transcriber's word errors against gen_text, or -1 when it had nothing to say. A transcriber that is busy with someone else's request, or reloading its weights, answers with nothing for a moment — so an empty answer is asked again, a few times, before it counts as no answer.

@ f5_synth_chunk F5Model m Vocos vc F5Voice v F5Vocab vocab s gen_text

@ f5_crossfade ( Vec f ) acc ( Vec f ) next i fade → v

Join next onto acc with a linear cross-fade over fade samples.

@ f5_synth F5Model m Vocos vc F5Voice v F5Vocab vocab s gen_text

: i F5_LEAD_WORDS 1

A line that opens with a one-word sentence — "Juuri. Seuraavaksi ne tuo…" — loses it: the model runs the reference straight into the second sentence and the first is never said, from every seed. Said on its own the same word comes out fine. So under --short-fix a one-word opening sentence is its own chunk, with its own duration estimate. Two-word openings ("Eipä kestä.", "Kiitos kutsusta.") were said as part of the line and are left in it. Returns where the rest begins, or 0 when the rule does not apply.

@ _f5r_lead_split s text → i

@ f5_synth_scored F5Model m Vocos vc F5Voice v F5Vocab vocab s gen_text

The utterance, chunk by chunk, each through the gate: retries more attempts for a chunk whose word error rate is over max_wer, and the score of what was kept added to score.

@ f5_limit_peak ( Vec f ) wave → v

Scale a waveform down, once, if it would clip on the way to 16 bits. The model's output is louder than its reference for some voices, and with target_rms off nothing else stands between it and the encoder.

@ f5_prepare_reference ( Vec f ) x i rate → ( Vec f )

The whole of preprocess_ref_audio_text's audio half.

@ f5_resolve_file s arg s ext → String

@ f5_resolve_vocab s arg s model_path → String

vocab.txt beside the checkpoint, unless one was named.

: i F5_MIN_SPLIT_WORDS 5

@ f5_strip_brackets s text → String

@ f5_split_sentences s text → ( Vec String )

Sentences, split at . ! ? followed by whitespace, with anything under five words merged into its neighbour — a two-word sentence given a duration estimate of its own is exactly the case the model fumbles.

@ f5_word_count s text → i

@ f5_trailing_pause_ms s text → i

Milliseconds of silence a line's own punctuation asks for.

@ f5_append_silence ( Vec f ) out i ms → v

@ f5_synth_line F5Model m Vocos vc F5Voice v F5Vocab vocab s text

@ f5_voice_rms F5Voice v__h → f

main.nu

packages/f5tts — F5-TTS, the flow-matching text-to-speech model, in pure NURL. CLI:

f5tts synth --model NAME --vocoder NAME --voice DIR --text "..." -o out.wav f5tts serve --model NAME --vocoder NAME [--addr H:P] f5tts tokens <vocab.txt> <file> one line of ids per line of the file f5tts chunks <file> --max N the text split the way F5-TTS splits it

API

@ main → i

sample.nu

packages/f5tts/src/sample.nu — the ODE that turns noise into a mel.

F5-TTS is a conditional flow-matching model: the network does not predict the mel, it predicts a VELOCITY, and the mel is what you get by starting at Gaussian noise and integrating that velocity from t=0 to t=1. Plain Euler, a fixed number of steps, no adaptive anything.

Two details decide whether it sounds like the reference voice:

with the reference audio and the text, once with neither — and pushes the answer away from the unconditional one: v = v + (v − v_null)·cfg. Both forwards go in ONE batch here, so the GEMMs are twice as tall rather than run twice.

towards zero by t + c·(cos(πt/2) − 1 + t) with c = −1, which spends most of the steps where the trajectory actually curves. It is the difference between 32 steps sounding like 32 steps and sounding like a hundred.

API

: f F5S_PI 3.14159265358979323846

@ f5_timesteps i steps f sway → ( Vec f )

The step schedule: linspace(0, 1, steps+1), swayed.

@ f5_noise i n i mel i seed → ( Vec f )

Gaussian noise for the whole mel, one draw per element.

@ f5_sample F5Model m ( Vec i ) ids i duration ( Vec f ) cond_mel

── the integration ─────────────────────────────────────────────────

cond_mel is the reference recording's mel, frames × mel_channels; the generated frames follow it in the same buffer and the model is free to overwrite them. out receives duration × mel_channels.

store.nu

packages/f5tts/src/store.nu — where a machine that has nothing keeps things.

Installed from the registry onto a fresh machine, f5tts has no models and no voices. It makes itself somewhere to put them:

~/.f5tts/voices/<id>/config.json the transcript ~/.f5tts/voices/<id>/reference.wav the recording the voice IS ~/.f5tts/models/<id>/… a checkpoint and its vocab.txt

$F5TTS_HOME moves the lot. Models named as Hugging Face references are not copied here at all — they go to the shared ~/.nurl/models cache through packages/hub, so whisper, embed and f5tts share one copy of anything they happen to share. This directory is for what the machine ITSELF has: voices somebody recorded, and checkpoints somebody trained or downloaded by hand.

API

@ f5_home → String

@ f5_voices_dir → String

@ f5_models_dir → String

@ f5_ensure_dirs → b

Creating a directory that is already there is not a failure, and this runs on every start: only a directory that is still ABSENT afterwards is worth saying anything about.

@ f5_id_ok s id → b

A name that may be joined onto a directory the server owns: a plain identifier, no separator, no dots, nothing a request could walk out of.

@ f5_voice_path s voices_dir s id → String

@ f5_voice_text s dir → String

The transcript a voice directory carries, or empty.

@ f5_voice_write s voices_dir s id s ref_text ( Vec u ) wav → String

Write a voice: the directory, the transcript, and the recording as it arrived. Returns an empty string on success or the reason on failure.

@ f5_voice_delete s voices_dir s id → b

@ f5_voice_list s voices_dir → ( Vec String )

Every voice directory that actually holds a recording.