mel.nu

packages/audio/src/mel.nu — STFT and the mel filter bank.

Every constant here was read out of transformers' audio_utils.py and feature_extraction_whisper.py, not remembered. The details that look like details are the ones that decide whether a speech model hears anything:

• the Hann window is PERIODIC — hanning(N+1) with the last sample dropped,

i.e. 0.5·(1 − cos(2πn/N)). The symmetric window (…/(N−1)) is a different window, and every frame would be subtly wrong.

• the signal is REFLECT-padded by n_fft/2 on both sides before framing

("center"), so frame k is centred on sample k·hop, not started there.

• the mel scale is SLANEY, not HTK: linear below 1 kHz (3f/200), log above.

HTK's 2595·log10(1+f/700) is the other convention, and it gives different filters.

• the filters are area-normalised, Slaney-style: 2/(f[k+2] − f[k]).
• whisper drops the LAST STFT frame, then floors the log at (max − 8) and

maps it to (x + 4)/4.

( mel_filters n_fft n_mels rate f_min f_max ) → ( Vec f ) (n_bins × n_mels) ( stft_power x plan window hop ) → ( Vec f ) frames × n_bins ( log_mel_whisper x n_fft hop n_mels rate ) → ( Vec f ) frames × n_mels

Data is row-major and flat: a spectrogram is one ( Vec f ), not a vec of vecs. It goes to a GPU next, and that is the layout a GPU wants.

API

: f MEL_PI 3.14159265358979323846

@ hz_to_mel_slaney f hz → f

Slaney mel: linear at 3f/200 below 1 kHz, logarithmic above.

@ mel_to_hz_slaney f mel → f

@ mel_filters i n_fft i n_mels i rate f f_min f f_max → ( Vec f )

The triangular filter bank, row-major (n_bins rows × n_mels columns) — the same orientation transformers uses, so a mel value is a dot product down a column.

@ hann_periodic i n → ( Vec f )

Periodic Hann: 0.5·(1 − cos(2πn/N)). NOT the symmetric one.

@ reflect_pad ( Vec f ) x i pad → ( Vec f )

Reflect-pad by pad samples on both sides: x[pad], x[pad-1], …, x[1] in front (the sample itself is not repeated) and the mirror image at the end. This is what "center=True" means, and it is why frame k is CENTRED on sample k·hop rather than starting there.

@ stft_power * FftPlan p ( Vec f ) x ( Vec f ) window i n_fft i hop → ( Vec f )

|STFT|², row-major frames × (n_fft/2+1). x is already padded; the caller owns the plan (an STFT runs the same length thousands of times).

@ log_mel_whisper ( Vec f ) x i n_fft i hop i n_mels i rate → ( Vec f )

The whisper log-mel spectrogram: exactly what WhisperFeatureExtractor produces, in the same layout (frames × n_mels, row-major).

@ pad_or_trim ( Vec f ) x i n → ( Vec f )

Pad with zeros (or trim) to exactly n samples — whisper always feeds its encoder 30 s, whatever the clip is.

---

main.nu

packages/audio/src/main.nu — the CLI.

audio info <file.wav> rate, channels, bits, duration audio mel <file.wav> -o mel.f32 whisper log-mel (80 × 3000), raw f32 LE audio resample <in.wav> <out.wav> --rate 16000 audio tone <out.wav> --rate 16000 a known signal, for tests

API

@ main → i

---

wav.nu

packages/audio/src/wav.nu — RIFF/WAVE, read and write.

The chunk layout is the whole format, and it is also the whole attack surface: every chunk announces its own size, and a file is free to lie about it. So the reader walks the chunks with a bounded cursor — a size that runs past the end of the file, a data chunk longer than the bytes that follow, a zero or absurd sample rate, a channel count of 0, a bit depth nobody has ever used: all of them are a clean error, none of them are an allocation sized by what the header claimed.

( wav_read path ) → !Wav String f32 samples, interleaved ( wav_read_mono_16k path ) → !( Vec f ) String what speech models want ( wav_write path samples rate ch ) → !v String 16-bit PCM ( wav_free w ) → v

Samples come back as f32 in [-1, 1], whatever the file stored them as: 8-bit unsigned, 16/24/32-bit signed PCM, or 32/64-bit IEEE float.

API

: i WAV_PCM 1

: i WAV_FLOAT 3

: i WAV_EXTENSIBLE 65534

: Wav

: Wav {
    ( Vec f ) samples  // interleaved
    i channels
    i rate
    i bits  // as stored in the file (informational)
}

@ wav_free Wav w → v

: WCur

: WCur {
    ( Vec u ) d
    i n
    i off
    b fail
}

A bounded cursor over the file bytes. Every read checks the budget against what is LEFT, never by forming off + k (which a hostile size would wrap).

@ wav_read s path → !Wav String

@ wav_mono Wav w → ( Vec f )

Average the channels. (Speech models want one channel; picking the left one throws away half of a stereo recording's information.)

@ wav_write s path ( Vec f ) samples i rate i channels → !v String

16-bit PCM, which every tool on earth reads.

---

resample.nu

packages/audio/src/resample.nu — sample-rate conversion.

Linear interpolation is the tempting one-liner, and it is wrong in a way that matters here: it is a lousy low-pass filter, so downsampling with it folds everything above the new Nyquist frequency back into the speech band as aliasing — exactly the frequencies a speech model is listening to.

So: windowed-sinc (Kaiser-ish, a Hann-windowed sinc), the standard honest answer. The kernel is band-limited to the LOWER of the two Nyquist frequencies, which is what makes downsampling safe.

( resample x from_rate to_rate ) → ( Vec f )

API

: f RS_PI 3.14159265358979323846

: i RS_ZEROS 16

Zeros are half the taps of a windowed sinc — 16 gives a stopband deep enough that the aliasing is far below the noise floor of any real recording.

@ resample ( Vec f ) x i from_rate i to_rate → ( Vec f )