mlp.nu

mlp — a trainable multi-layer perceptron in pure NURL.

Dense layers with ReLU hidden activations and a linear output, trained with minibatch Adam on squared loss — the same recipe as sklearn's MLPRegressor, and deliberately faithful to it:

Glorot uniform init: W, b ~ U(−√(6/(fan_in+fan_out)), +√(…)),

drawn from a seedable PRNG (std/rng) — a fixed seed gives the same network on every platform.

squared loss with L2: grad = (AᵀΔ + α·W) / B per batch of B.
Adam with bias correction: lr_t = lr·√(1−β2ᵗ)/(1−β1ᵗ).
batch_size = min(200, n) — sklearn's "auto".
early stopping: a validation split (val_frac) is held out once,

the epoch loop stops after n_no_change epochs without the validation MSE improving by > tol, and the BEST weights seen are restored (not the last). With early_stop off the same patience applies to the training loss, mirroring sklearn. (sklearn scores R² on the validation set; for a fixed split maximising R² is minimising MSE, so the criterion is the same up to tol's scale.)

An autoencoder is just ( mlp_train m X n d X d cfg ) — input as target.

Layout: one flat weight buffer + one flat bias buffer for the whole network (struct-of-arrays, like iforest's node arena). Layer l maps sizes[l] → sizes[l+1]; its weights are row-major n_out×n_in at w_off[l], its biases at b_off[l]. Hot loops read through raw pointers (vec_data) and write with vec_set — the fft.nu idiom.

Persistence: mlp_save/mlp_load round-trip through JSON with every f64 stored as its bit pattern (i64) — exact, platform-independent restore (nurl_str_float renders ~6 significant digits, so decimal text would silently perturb a trained model).

API

`: Mlp`

: Mlp {
    i n_layers  // weight layers = len(sizes) − 1
    ( Vec i ) sizes  // neuron counts, input first, output last
    ( Vec i ) w_off  // offset of layer l's weights in `w`
    ( Vec i ) b_off  // offset of layer l's biases in `b`
    ( Vec i ) a_off  // offset of layer l's activations in a scratch
    i n_w  // total weight count
    i n_b  // total bias count
    i n_a  // total activation count (all layers, one sample)
    ( Vec f ) w
    ( Vec f ) b
    // Adam state, same shapes as w / b
    ( Vec f ) mw
    ( Vec f ) vw
    ( Vec f ) mb
    ( Vec f ) vb
    i t  // Adam step counter
}

`: MlpCfg`

: MlpCfg {
    f lr  // Adam initial learning rate (sklearn: 0.001)
    f alpha  // L2 penalty (sklearn: 0.0001)
    i max_iter  // epochs (sklearn: 200; sklearn AE recipes use 500)
    i batch  // minibatch size; 0 = min(200, n) ("auto")
    b early_stop  // hold out a validation split and stop on plateau
    f val_frac  // validation fraction (sklearn: 0.1)
    i n_no_change  // patience in epochs (sklearn: 10)
    f tol  // minimum improvement (sklearn: 1e-4)
    i seed  // PRNG seed (init + shuffles + the val split)
    b verbose  // print per-epoch losses
}

`: MlpTrain`

: MlpTrain {
    i n_iter  // epochs actually run
    f loss  // final training MSE (plain mean squared error)
    f best_val  // best validation MSE (early_stop) or best train MSE
    b stopped_early  // the patience tripped before max_iter
}

What mlp_train reports back.

`: MinMax`

: MinMax {
    i n_cols
    ( Vec f ) lo
    ( Vec f ) hi
}

Min–max scaler: x' = (x − lo) / (hi − lo), zero-range columns → 0.

`@ _mlp_fget ( Vec f ) v i idx → f`

`@ _mlp_iget ( Vec i ) v i idx → i`

`@ mlp_new ( Vec i ) sizes i seed → Mlp`

A fresh network with Glorot-uniform weights. sizes lists every layer width, input first (e.g. [8 64 32 64 8] for the classic autoencoder); the caller keeps ownership of sizes (it is copied).

`@ mlp_cfg_default → MlpCfg`

`@ mlp_free Mlp m → v`

`@ mlp_predict Mlp m ( Vec f ) x → ( Vec f )`

Predict one row (a ( Vec f ) of the input width) → an OWNED output vec.

`@ mlp_mse Mlp m ( Vec f ) X i n i d ( Vec f ) Y → f`

Mean squared error of the network over rows X (n×d row-major) against targets Y (n×dout row-major): mean over every output element.

`@ mlp_train Mlp m ( Vec f ) X i n i d ( Vec f ) Y i dout MlpCfg cfg → MlpTrain`

Train on X (n×d, row-major) against Y (n×dout). For an autoencoder pass X for Y and d for dout. Returns the training report; the model is updated in place (best-validation weights when early_stop is on).

`: MlpFit`

: MlpFit {
    Mlp model
    MlpTrain report
}

`@ mlp_fit ( Vec i ) sizes ( Vec f ) X i n i d ( Vec f ) Y i dout MlpCfg cfg i restarts → MlpFit`

`@ mlp_save Mlp m → String`

Serialise the trained network (weights + biases; Adam state is NOT persisted — a loaded model predicts, or fine-tunes from a fresh optimiser). Returns an OWNED String of JSON.

`@ mlp_load s src → ?Mlp`

Rebuild a network from mlp_save output. F on malformed input.

`@ minmax_fit ( Vec f ) X i n i d → MinMax`

`@ minmax_apply MinMax mm ( Vec f ) X i n → v`

Scale rows in place: x' = (x − lo) / (hi − lo); zero-range columns → 0.

mlp 0.1.0 API