service.nu

anomaly/service.nu — HTTP/JSON service (milestone M6).

Thin: parse JSON → call the library → serialise. The routes and the request/response shapes mirror the Python reference service so existing dashboards keep working:

POST /detect/<model> ingest + verdict (202 warming) POST /detect_only/<model> score only, no state change GET|POST /force_train/<model> retrain now POST /detect_anomalies batch-score a CSV file GET /models/dynamic list models + metadata GET /models/dynamic/<model>/metadata metadata (+ autoencoder state) PUT /models/dynamic/<model>/metadata edit schedule / version configs GET /models/dynamic/<model>/data recent points (?limit=N&from=&to=&last=&fields=) GET /models/dynamic/<model>/anomalies scored ring (cached, filtered) GET /models/dynamic/<model>/calibration alert rates vs margins POST /models/dynamic/<model>/import a CSV/JSON/JSONL file of history POST /models/dynamic/<model>/fork a new model trained on a slice of this one's history POST /models/dynamic/<model>/reset drop data+forests, keep name DELETE|GET /delete_model/<model> delete entirely PUT /api/dynamic/<model>/schedule retraining schedule POST /api/dynamic/<model>/finetune set margins from a target alert rate POST /train/autoencoder/<model> train the autoencoder version POST /mcp the same API for a language model (src/mcp.nu) GET /.well-known/oauth-protected-resource[/mcp] where /mcp callers get a token

Model names must match ^[a-zA-Z0-9_]+$ (400 otherwise, same message as the reference). Divergence from the reference: /detect_anomalies scores the CSV with a self-trained stateless forest (the reference's separate "static model" family doesn't exist here); passing model_name is a 400.

When a web root is configured (anomaly_service_set_webroot), the router also serves the self-contained dashboard pages from disk:

GET / | /modelmanager.html | /modeltrainer.html | /visualize.html | /anomalies.html | /admin.html | /oauth/callback | /auth.js

With no web root these routes 404 and the service is API-only.

The router is exposed separately from the socket (anomaly_service_router

• router_handle) so every route is testable without networking.

API

: ~ s g_an_root .``

Store root used by every handler; set once before serving.

@ anomaly_service_set_root s root → v

: ~ s g_an_webroot ``

Directory holding the dashboard HTML (modelmanager.html, etc). Empty = static serving disabled (API-only). Set once before serving.

@ anomaly_service_set_webroot s root → v

: i AZ_GATE_OK 0

: i AZ_GATE_UNAUTH 401

: i AZ_GATE_FORBID 403

: Gate

: Gate {
    b allowed
    i status
    Principal who
    b creating  // the named model does not exist yet; this call would make it
}

: ~ i g_an_lock 0

The service lock. The server runs a small worker pool so that a handler which WAITS — analyze, for its job — can let go of the service while it does; every other handler holds the lock for its whole run, so the store, the GPU singleton and the authorization layer see one request at a time, exactly as under the single-threaded server. 0 = no pool (tests drive the router directly), and the release/acquire pair is a no-op.

: i AN_ANALYZE_WAIT_DEFAULT 10

POST /api/analyze — the body is the file. Query: format (csv, json, jsonl, fmi; empty = detect), time, tz, calendar, clock as for an import, name to label the result, and wait — how many seconds to hold the call for the result (10 by default, 60 at most, 0 = return the task at once). A result that is not ready in time answers 202 with the task to ask again for.

: i AN_ANALYZE_WAIT_MAX 60

@ anomaly_service_router → Router

@ anomaly_serve s host i port → i

Serve until the listener errors. Returns a process exit code. Serve the routes over the http package's App facade: it owns the bind + keep-alive loop, a graceful SIGINT/SIGTERM shutdown, and turns a handler panic into a 500 (rather than dropping the connection). The router itself is still built by anomaly_service_router, so every route stays drivable without a socket in the test suite.

---

dynamic.nu

anomaly/dynamic.nu — dynamic streaming models (milestones M4 + M5).

The headline feature: a named model that is created on first use, ingests one raw JSON point at a time, and trains itself once enough history has accumulated. Each model keeps:

• a bounded ring of raw points (max_points, default 150 000) persisted

as data.jsonl — raw records, so retrains learn new categories;

• schedule-driven retraining: whenever the lifetime point count

(n_seen, monotonic — unaffected by ring eviction) reaches the next training mark, every enabled version retrains; the mark then advances by schedule.below_max (default 50) or, once the ring is full, schedule.at_max (default 1000);

• one forest per enabled version (M5): each version trains on its own

time window of the ring (window_min minutes back from "now", or the last window_pts points), falling back to the most recent min_points rows when the window is too thin;

• one shared scaler, fit over the full ring at each train.

Verdicts follow the reference service: a point is anomalous if ANY enabled version flags it (decision_function <= -margin); the reported score is the most severe (lowest) decision_function. Before min_points points (default 50) the model is warming up: ready = false, no verdict.

Every public entry point has an _at variant taking now in unix seconds — the injectable clock that makes window filtering and stamped timestamps reproducible in tests. The plain variants read the wall clock.

API

: VerVerdict

: VerVerdict {
    String vvname
    b anomaly
    f score
    f margin
    f cfg_margin
}

One version's slice of a verdict. margin is the EFFECTIVE absolute band the score was compared against (the invariant score <= -margin ⇒ anomaly holds for every version); cfg_margin is the number stored in the metadata. They differ only for the autoencoder, whose configured margin is relative to its reconstruction threshold — see ANOM_AE_MARGIN.

: Verdict

: Verdict {
    b ready
    b anomaly
    f score
    ( Vec VerVerdict ) versions
}

The aggregate verdict for one point (SPEC §5.4).

: Model

: Model {
    Store store
    String mname
    * Meta meta
    ( Vec String ) lines
    ( Vec i ) times
    ( Vec VerModel ) forests
    Scaler sc
    AeModel ae
    i next_train_at
    i min_points
    i max_points
}

A live dynamic model. Obtain with model_open, release with model_free.

@ verdict_free Verdict vd → v

@ _an_line_ts s line → i

Ingest timestamp of a stored data.jsonl line (0 if unparsable).

@ model_open_at Store st s name i now → *Model

@ model_open Store st s name → *Model

@ model_set_version_window * Model mo s vname i wsize i sstep → b

Set a version's sliding-window geometry (timevector). Takes effect at the NEXT retrain — detect derives the live window from the trained forest's width, so a config change can never desync scoring.

@ model_last_ts * Model mo → i

The newest stored stamp, or 0 on an empty ring.

@ model_next_tick * Model mo → i

The tick the NEXT point gets on the count clock: one past the newest.

@ model_now * Model mo → i

"Now" for this model: the wall clock, or on the count clock the newest tick — the moment the last point arrived is the only present it has.

@ model_free * Model mo → v

@ model_set_limits * Model mo i min_pts i max_pts → v

Test hook: shrink the warm-up / ring limits so eviction and scheduling are exercisable without 150 000 points.

@ model_metadata * Model mo → *Meta

@ model_n_points * Model mo → i

@ model_is_trained * Model mo → b

@ model_force_train_at * Model mo i now → i

Retrain every enabled version from the ring, as of now. Re-encodes the raw ring (so new categories/columns learned since the last train enter the feature order), refreshes the authoritative feature order, refits the shared scaler over the full ring, then trains each version on its window. Returns the number of ring points used (0 = not enough data, no change).

@ model_force_train * Model mo → i

@ model_train_autoencoder * Model mo ( Vec i ) hidden f contamination → String

Train the autoencoder from the ring: encode + project the raw points (the same pass model_force_train_at runs, minus the standardising scaler — the AE recipe MinMax-scales after anomaly filtering), then hand the matrix to ae_train_matrix. hidden empty → 64-32-64. Explicit by default; with schedule.autoencoder on, every forest retrain repeats it with the same layout and pre-filter (see __an_retrain_ae). Returns the error text ("" = success).

@ model_train_autoencoder_at * Model mo ( Vec i ) hidden f contamination i now → String

@ model_ingest_at * Model mo Json raw i now → !Verdict String

Add one raw point: encode (learning), stamp timestamp, append to the ring (evicting the oldest at capacity), persist, retrain if the schedule says so, then score it. Errors (bad numeric / timestamp values) leave the model completely untouched.

@ model_ingest * Model mo Json raw → !Verdict String

@ model_detect_only * Model mo Json raw → !Verdict String

Score without ingesting: no metadata learning, no ring append, no retrain, no disk writes. Unknown columns/categories project to zeros.

: ImpRow

: ImpRow {
    i ir_ts
    String ir_line
}

: ImportReport

: ImportReport {
    i accepted
    i rejected
    i stored  // points in the ring afterwards
    b trained
    String err  // non-empty ⇒ nothing was imported
    ( Vec String ) notes
}

@ import_report_free ImportReport r → v

@ model_import_at * Model mo ( Vec Json ) recs i now → ImportReport

Import recs into the model. Records keep their own timestamp when they carry one; the rest are placed at now. On the count clock a record's own stamp is ignored: the file's order is its time, and the rows take the ticks after the newest stored point, one each. Returns what happened.

@ model_import * Model mo ( Vec Json ) recs → ImportReport

: CalVer

: CalVer {
    String cvname
    f cur_margin  // the configured margin (own units)
    i n  // rows this version had a verdict for
    i flagged  // rows the current margin flags
    f worst  // the most negative decision value
    f median
    ( Vec f ) dfs  // ascending
}

One version's decision values over the calibration window.

: CalReport

: CalReport {
    ( Vec CalVer ) items
    i from_ts  // resolved window (0 = unbounded)
    i to_ts
    i n_rows  // rows scored
    i agg_flagged  // rows some enabled version flagged at the current margins
}

@ cal_free CalReport rep → v

@ round_sig f x i digits → f

Round to digits significant decimal digits, through an exact integer mantissa and a power of ten so the result is the double nearest to the short decimal (the same double the literal "0.106" parses to — division by an exactly representable power of ten is correctly rounded), and therefore prints as that short decimal.

@ round_sig_dir f x i digits i dir → f

dir picks the rounding: 0 nearest, -1 toward zero, +1 away from zero (on the magnitude; the sign is restored afterwards).

@ cal_flagged_at CalVer cv f margin → i

Rows the version would flag at margin: count of dfs <= -margin.

@ cal_margin_for_rate CalVer cv f rate → f

The margin at which a fraction rate of the window is flagged: the k-th most negative value, k = round(rate·n), sits exactly on the line (so it is flagged, and the (k+1)-th is not); rate 0 asks for a margin just above the worst point. The exact value is then rounded to the FEWEST significant digits (2 to 6) that still flag the same count, give or take a tenth of it — a margin is a setting a person reads and retypes, and "0.13" is one where the data allows it, "0.1284" where the decision values are packed too densely for fewer digits. Never negative: a negative margin would flag points the forest itself calls normal, and a rate the data cannot supply is answered by the honest post-rounding count, not by a margin below zero.

@ model_calibrate * Model mo i from_ts i to_ts → CalReport

Score every ring row in [from_ts, to_ts] (0 = unbounded) through the live verdict path and collect each version's decision values. Rows are scored as of their own ring position, exactly as the scan does, so a timevector window never sees the future.

: i ANOM_CAL_WINDOW 86400

The window fine-tune and calibration default to: the newest 24 hours of stored data, anchored on the newest stored point rather than the clock, so a model whose feed stopped still calibrates on its last day.

@ model_window_from_last * Model mo i to_ts i last → i

Resolve (from, to, last) the way the HTTP layer spells it: last seconds back from to, or from the newest stored point when to is unbounded.

: FtVer

: FtVer {
    String ftname
    f old_margin
    f new_margin
    i n  // rows in the window with a verdict
    i before  // flagged at old_margin
    i after  // flagged at new_margin
    f worst
    b applied  // F on a dry run, or when the version was filtered out
    i ft_from  // the window's lower bound this version was tuned over
    i ft_n_rows  // ring rows inside that window
}

One version's fine-tune outcome.

: FineTuneReport

: FineTuneReport {
    ( Vec FtVer ) items
    f rate
    i from_ts
    i to_ts
    i n_rows
    b applied
}

@ finetune_free FineTuneReport rep → v

@ model_finetune_at * Model mo f rate i from_ts i to_ts b apply ( Vec String ) only → FineTuneReport

Set every enabled, trained version's margin so that a fraction rate of the window [from_ts, to_ts] is flagged, or only report what would change when apply is F. only (empty = all) restricts which versions are written; the rest are still reported, unapplied, so a dry run and a partial apply show the same picture. Margins are written in the version's own units, rounded to three significant digits, and take effect at the next detect.

@ model_version_from * Model mo s vname → i

The lower bound of a version's OWN window, anchored on the newest stored point: window_min minutes back for a forest version, window_size points back for timevector, and the whole ring (0) for the autoencoder, whose training set is the whole ring too.

@ model_finetune_own * Model mo f rate b apply ( Vec String ) only → FineTuneReport

Fine-tune every version over ITS OWN window — the one it trains on — so short_term's margin answers for the last three hours and seasonal's for the last ninety days, each at the same rate. One calibration per version; the report's window is the widest of them.

: f ANOM_FT_RATE 0.01

The one-call form: 1 % of the last 24 hours, applied to every version.

@ model_last_span * Model mo i last → i

A last as a caller says it — seconds on a time clock, points on a count clock — as the span model_window_from_last takes. N points back from the newest is N ticks INCLUDING it, so the span is one short of N whole ticks.

@ model_default_last * Model mo → i

The default window: a day, or its worth of points (1440) on a count clock.

@ model_finetune * Model mo → FineTuneReport

: WholeTrain

: WholeTrain {
    Json margins  // version name → decision margin, as fine-tune set it
    Json notes  // strings: what could not be done, and why
}

@ whole_train_free WholeTrain w → v

@ model_train_whole * Model mo f rate → WholeTrain

: ScoredPt

: ScoredPt {
    i sp_idx  // ring index
    i sp_ts  // ingest timestamp (unix seconds)
    f sp_score  // aggregate decision_function (the most severe version)
    b sp_anomaly
    i sp_present  // bitmask over ScanOut.vnames: versions that had a verdict
    i sp_flagged  // bitmask over ScanOut.vnames: versions that flagged it
}

One scored ring point.

: ScanOut

: ScanOut {
    ( Vec ScoredPt ) pts
    ( Vec String ) vnames
    i epoch
    i total  // ring size
    i considered  // rows inside the requested time window
    i hits  // verdicts answered from the cache
    i misses  // verdicts computed this call
    i anomalies  // anomalous rows among `considered`
}

@ scan_free ScanOut so → v

@ model_scan_versions * Model mo → ( Vec String )

The canonical version order a scan's bitmasks are indexed by: every enabled version that could produce a verdict, in metadata order. It is stable within an epoch by construction — anything that adds, removes or toggles a version bumps the epoch — and it is written into the cache so a mismatch is caught rather than silently misread as different versions.

@ model_scan_at * Model mo i from_ts i to_ts i limit b force → ScanOut

Score every ring point whose timestamp falls in [from_ts, to_ts] (0 = unbounded on either side), newest-last, capped at limit rows (<= 0 = no cap) taken from the END of the window. force recomputes even when the cache is warm — the escape hatch for verifying the cache itself, never needed for correctness.

@ model_scan * Model mo i from_ts i to_ts i limit b force → ScanOut

@ model_point_json * Model mo i at → ?Json

The raw stored record at a ring index, parsed (None when out of range or no longer parsable).

: AeContrib

: AeContrib {
    String ac_name
    f ac_err
    f ac_share
    f ac_value
    f ac_expected
}

One feature's share of an autoencoder reconstruction error, with the value the point carried and the one the autoencoder expected of it given the other features (raw units — the MinMax undone).

@ ae_contrib_free ( Vec AeContrib ) xs → v

@ model_ae_contrib * Model mo Json raw i topk → ( Vec AeContrib )

The topk features carrying the most of a point's reconstruction error, largest first, with each one's share of the total. This is the answer the forests structurally cannot give: the autoencoder's per-feature error is how badly that feature failed to be predictable FROM THE OTHERS, so the top entries name the broken relationship rather than the extreme value. Empty when the model has no trained autoencoder.

@ model_reset * Model mo → v

Drop all data and trained forests but keep the model's name, schedule and version configs. Learned columns/categories/features/scaler reset.

@ model_delete Store st s name → b

Delete a model from the store entirely (the Model handle, if any, should be freed separately with model_free).

@ model_set_margin * Model mo s vname f margin → b

Set one version's decision margin in the metadata (persisted, effective immediately at scoring — no retrain needed). Returns F for an unknown version name.

@ model_set_schedule * Model mo i below_max i at_max → v

Update the retraining schedule (persisted immediately).

@ model_set_version_enabled * Model mo s vname b on → b

Turn one version on or off (persisted immediately). Disabling drops the version's forest, so its verdict is gone from the very next detect and re-enabling it costs a retrain. The autoencoder version is only muted: its net carries its OWN frozen feature order, so it stays valid across retrains and is far too expensive to throw away on a checkbox. Returns F for an unknown version name.

@ meta_editable_fields → Json

Apply an editable-metadata patch:

{ "alias": "boiler room", "schedule": { "below_max": N, "at_max": N, "autoencoder": bool }, "max_data_points": N, "versions": { "<name>": { <any VerCfg field> }, ... }, "replace_versions": bool }

All top-level keys are optional but at least one must be present. The learned parts of the metadata (columns, categories, feature order, scaler) are never taken from the client — see prep.nu. Lowering max_data_points below the current fill evicts the oldest points and rewrites the log before returning, so the cap holds at once instead of converging one point per ingest. Per-version fields split two ways: enabled and decision_margin bite at the very next detect, the geometry and forest-size fields at the next retrain. Returns "" on success, else the reason (400-worthy). The top-level keys model_apply_meta_patch accepts, published with every metadata response so a client does not have to keep its own copy of the list. The dashboard kept one, and it went stale the moment max_data_points became editable: the field was patchable through the API and invisible in the JSON editor that exists to reach it. One list, named here beside the code that reads the patch, is what stops that from happening again — the editor is generated from it.

Deliberately NOT part of meta_to_json: that is also the on-disk meta.json writer, and a service-shaped descriptor has no business in the stored file.

@ model_apply_meta_patch * Model mo Json patch → String

---

imptime.nu

anomaly/imptime.nu — where a file keeps its clock, and how to read it.

A stream stamps every point at the moment it arrives. A file has no such moment: its clock is in the data — a timestamp column, an ISO string under some other name, a Unix number, or a date spread over Vuosi, Kuukausi, Päivä and Aika the way a weather-service export writes it. This file reads all of those, best effort, and says which one it read, so the person importing can confirm or correct the guess before a single row lands.

Two operations:

import_inspect parsed rows → JSON: every column with what its values look like, and a proposal for the time — one column, a set of part columns, or nothing. import_time_apply rows + a plan → every row gets timestamp (Unix seconds) from the plan; the source columns are dropped so a year or a clock does not become a feature; and, on request, a time ISO string is left behind for the calendar features (hour, day, month, weekday) the preprocessing layer derives.

Naive stamps — a clock without an offset, which is what a database TIMESTAMP WITHOUT TIME ZONE, a spreadsheet and a weather export all write — are read in the timezone the caller names, defaulting to the server's local one: the file came from the place the service runs.

API

: i ANOM_TZ_LOCAL -1000000

The caller's "use the server's local zone" sentinel for a tz offset.

: i ANOM_INSPECT_ROWS 5000

Rows looked at when describing a file. A million-row file is described by its first five thousand; the plan then applies to all of them.

: i STAMP_NONE 0

What one cell turned out to be.

: i STAMP_DATETIME 1 // a date and a time of day

: i STAMP_DATE 2 // a date alone (midnight)

: i STAMP_UNIX 3 // a Unix number — seconds, ms, µs or ns by magnitude

: i STAMP_CLOCK 4 // a time of day alone, HH:MM[:SS]

: ImpStamp

: ImpStamp {
    i kind
    i secs
    b zoned
}

secs is Unix seconds when zoned (the text carried an offset, or was a Unix number); otherwise the civil fields read as if they were UTC, for the caller to shift into the zone it was told. For STAMP_CLOCK it is seconds after midnight.

: __ItNum { i val i len }

@ imp_stamp_of_text s raw → ImpStamp

One cell of text as a stamp. Formats, in the order they are tried:

2026-08-29T00:10:00Z, 2026-08-29 00:10:00+03:00, 2026-08-29 00:10, 2026-08-29 ISO 8601 / RFC 3339, SQL TIMESTAMP with or without a zone, date alone 2026/08/29 00:10 the same with slashes 29.8.2026 00:10, 29.8.2026 day first — dots are European 29/08/2026, 08/29/2026 slashes: day first unless the second number cannot be a month 20260829, 20260829T001000 compact 1756422600, 1756422600000 Unix seconds / ms / µs / ns Sat, 29 Aug 2026 00:10:00 +0300 RFC 2822 and HTTP dates 00:10, 00:10:00 a time of day alone (STAMP_CLOCK)

@ imp_stamp_of Json v → ImpStamp

One JSON cell as a stamp: a number is Unix, a string is text.

@ imp_instant_of_text s raw i tz → i

One instant from one piece of text, as a Unix second: an ISO 8601 date or date-time (a bare one is read in tz, ANOM_TZ_LOCAL for the server's zone), a Unix number in seconds, milliseconds, microseconds or nanoseconds (told apart by magnitude), or a compact 20260901[120000]. 0 when the text is not a moment — a time of day alone is not one.

@ imp_span_of_text s raw → i

A span from text: "90s", "15m", "24h", "7d", "2w", or a bare number of seconds. 0 when unreadable or not positive.

@ imp_tz_of Json spec → i

A tz spelling: "local", "utc", "Z", "+03:00", "+0300", "+03", or a number of seconds east. Unreadable → local.

: i ROLE_STAMP 0

Does name (normalised) mean one of the roles? 2 = a strong name nobody would use for anything else, 1 = a short one that might be "minimum" or "seconds" or a coordinate, 0 = no.

: i ROLE_YEAR 1

: i ROLE_MONTH 2

: i ROLE_DAY 3

: i ROLE_HOUR 4

: i ROLE_MINUTE 5

: i ROLE_SECOND 6

: i ROLE_DATE 7

: i ROLE_CLOCK 8

: ImpCol

: ImpCol {
    String cname
    String norm
    i filled
    i n_num
    i n_text
    i n_dt  // text that reads as a date and time
    i n_date  // text that reads as a date alone
    i n_clock  // text that reads as a time of day alone
    i n_unix  // numbers that could be Unix stamps
    i n_year  // numbers 1900..2100
    i n_month  // numbers 1..12
    i n_day  // numbers 1..31
    i n_hour  // numbers 0..23
    i n_minsec  // numbers 0..59
    String sample
}

@ import_time_propose ( Vec ImpCol ) cols → Json

The plan: { mode: column|parts|none, column?, parts?: {year, month, day, hour, minute, second, date, clock}, confidence: high|guess|none, reason }. Every named column exists and fits its role.

: ImpTimeResult

: ImpTimeResult {
    i stamped
    i failed  // rows the plan could not read a time from — dropped
    String first_fail  // what the first of them looked like
}

@ imp_time_result_free ImpTimeResult r → v

@ import_time_apply ( Vec Json ) rows Json plan b calendar i tz → ImpTimeResult

Rewrite rows in place under plan. With mode none nothing changes. A row whose time cannot be read is removed: a history point with no place in history is not a point this model can hold.

: ImpPlan

: ImpPlan {
    Json plan
    String err
}

Turn a request ({ mode: auto|none|column|parts, column?, parts? }) into a concrete plan against these columns. auto becomes the proposal; a named column that does not exist is an error (non-empty err).

@ imp_plan_free ImpPlan p → v

@ import_time_plan ( Vec ImpCol ) cols Json spec → ImpPlan

@ import_inspect ( Vec Json ) rows Json spec i tz → Json

{ columns: [{ name, kind, filled, sample }], time: <plan + sample>, hints: { role: [names] } }. time.sample is the first row's time under the plan, as ISO in the zone, so a person can check the guess against the file with their own eyes.

---

config.nu

anomaly/config.nu — the configuration file.

Everything the service can be told is settable three ways, and they layer in one fixed order:

command-line flag > environment variable > config file > default

The file is the persistent baseline a deployment writes once; the environment is what a container or a unit file overrides for one run; a flag is what a person types to override both. Nothing here reads the environment or the command line — main.nu owns that layering, so the precedence lives in one place instead of being re-decided per setting.

The file is TOML, because that is already the project's format:

[auth] enabled = true issuer = "https://login.microsoftonline.com/<tenant>/v2.0" client_id = "<application (client) id>" audience = "api://<application (client) id>" # optional open_ingest = true

[service] addr = "0.0.0.0:8811" webroot = "/usr/share/anomaly/static" public_url = "https://anomaly.example.com" # behind a proxy

Deliberately NOT settable here: the store directory. The file is looked for inside the store, so a store key would be a file relocating the directory it was just found in — a loop with no honest answer. It stays --store / $ANOMALY_HOME.

API

: AnomalyConfig

: AnomalyConfig {
    b loaded
    String cpath  // where it came from; empty when nothing was loaded
    String cerr  // parse failure text; empty when fine
    TomlValue root
}

@ config_empty → AnomalyConfig

@ config_free AnomalyConfig c → v

@ config_load s path → AnomalyConfig

Read and parse path. A file that does not exist is not an error — the common case is having none — but a file that exists and does not parse IS one: silently ignoring a config file someone wrote is how a service comes up unconfigured and nobody can see why.

@ config_loaded AnomalyConfig c → b

@ config_path AnomalyConfig c → s

@ config_error AnomalyConfig c → s

@ config_has AnomalyConfig c s key → b

Is the dotted key present at all? The difference between "absent" and "set to empty" is what lets a lower layer show through.

@ config_str AnomalyConfig c s key s dflt → String

@ config_bool AnomalyConfig c s key b dflt → b

@ config_str_list AnomalyConfig c s key s dflt → String

A string array, joined with commas — the form a global can hold. An absent key, a non-array, or an array with a non-string in it all yield the default, so a mistyped list demotes to it rather than to a half-read one.

@ config_find s explicit s store → String

Where the configuration file is looked for, in order:

1. explicit — --config FILE, or $ANOMALY_CONFIG
2. <store>/anomaly.toml
3. /etc/anomaly/anomaly.toml

An explicit path is returned whether or not it exists, so a typo in --config surfaces as "that file does not parse / is not there" rather than as the service quietly falling back to a different file.

---

model.nu

anomaly/model.nu — the per-point scoring core (milestone M2).

Wraps the iforest package's forest behind the M1 preprocessing layer and the scikit-learn decision conventions the reference service uses:

score_samples(x) = -iforest_score(x) (more negative = worse) decision_function(x) = score_samples(x) - offset_ offset_ = -0.5 for contamination "auto" = percentile(score_samples of the training set, 100 * contamination) otherwise predict(x) == -1 ⇔ decision_function(x) < 0 version verdict : decision_function(x) <= -decision_margin

A VerModel is one trained forest + its offset and margin — the unit that M4/M5 instantiate per time window. Training and the bulk/batch scoring paths (GPU-accelerated when available) live in src/score.nu; this file is the small, dependency-light core they build on.

API

: i ANOM_SEED 42

The reference's fixed random_state: a given training set always yields a byte-identical forest, hence identical scores, on every platform.

: VerModel

: VerModel {
    String vname
    IForest forest
    f offset
    f margin
    i n_cols
    b trained
}

One trained model version: forest + decision offset + margin.

@ anom_decision VerModel vm ( Vec f ) scaled_point → f

decision_function of an already-standardised point.

@ anom_decision_row VerModel vm ( Vec f ) scaled i r → f

decision_function of row r of an already-standardised matrix.

@ anom_is_anomaly VerModel vm f df → b

The per-version verdict convention: below (or at) the margin ⇒ anomaly.

@ anom_vermodel_free VerModel vm → v

---

orgfiles.nu

src/orgfiles.nu — an organisation's storage folder

Every organisation has a folder of files beside its database:

<root>/orgs/<org>/files/<name>

Analyses drop their results there; the API lists and serves them to the organisation's members. A file can also be handed out as a link that carries its own permission — an HMAC over (org, name, expiry) under a secret the service generates once — so a result too large to return inline can be fetched by whoever holds the link, with no sign-in. The link names one file for a bounded time and nothing else; the secret never leaves the store.

File names are the public identifier, so they are kept to one safe alphabet: letters, digits, ., _, -, not starting with a dot, at most 128 bytes. That rules out path tricks and URL escaping alike.

API

: i OF_NAME_MAX 128

: i OF_LINK_TTL_DEFAULT 604800 // a week

: i OF_LINK_TTL_MAX 2592000 // 30 days

: OfState

: OfState {
    String root
    String secret
}

What the module keeps for the life of the process: the store root (the models' directory — the organisation folders live under <root>/orgs, beside the organisation databases) and the link-signing secret. Both are OWNED copies in one heap block a global points at, so a caller may free the root it passed, and the block stays reachable — a global that held only a data pointer would leave the String header unreachable, which is a leak in every accounting.

: ~ i g_of_state 0

@ orgfiles_set_root s root → v

@ orgfiles_dir s org → String

@ orgfiles_tasks_dir s org → String

@ orgfiles_name_ok s name → b

One safe alphabet for file names: [A-Za-z0-9.-], no leading dot, at most OFNAME_MAX bytes.

@ orgfiles_safe_name s raw → String

A caller-chosen label, made into a safe name: every byte outside the alphabet becomes _, a leading dot too, and the result is cut to fit. Empty in, empty out — the caller picks a default.

@ orgfiles_path s org s name → String

: OrgFile

: OrgFile {
    String name
    i size
    i mtime
}

@ orgfiles_free ( Vec OrgFile ) xs → v

@ orgfiles_list s org → ( Vec OrgFile )

Every file in the folder that carries a valid name, by name. Anything else in the directory — a temp file mid-write, a stray dotfile — is not a file the API ever handed out and is not listed.

@ orgfiles_stat s org s name → OrgFile

The stat of one named file, or size -1 when there is no such file.

@ orgfiles_delete s org s name → b

@ orgfiles_write s org s name ( Vec u ) data → b

Write a file into the folder atomically (tmp + rename), so a reader never sees a half-written result.

@ orgfiles_sign s org s name i exp → String

@ orgfiles_verify s org s name i exp s sig i now → b

@ orgfiles_link s org s name i exp → String

The link's path + query, relative to the service root.

@ orgfiles_content_type s name → s

Content type from the extension: the results are JSON, a caller may also have dropped CSV there; the rest is bytes.

---

importer.nu

anomaly/importer.nu — a file of history becomes a stream of points.

The service is built around one point at a time arriving from a producer. Importing is the other direction: a file that already holds the history, turned into the same records the ingest path takes, so nothing downstream has to know where a point came from.

Three shapes, because these are the three a data file actually arrives in:

csv a header row naming the columns, one row per point. Cells that parse as numbers become numbers; everything else stays a string, which the preprocessing layer already knows what to do with (categories, ISO-8601 timestamps → calendar features). json an array of objects, or an object with the array under data, points or rows — the three spellings an export tool picks. jsonl one object per line. What this service's own /data route emits, so a model can be moved by exporting and importing it.

auto sniffs: a body whose first non-space character is [ or { is JSON or JSONL, anything else is CSV. Detection is offered because a browser upload rarely knows its own format, and is never mandatory — naming the format explicitly always wins.

A row that cannot be read does not stop the import: it is counted and the first few are described. A file of ten thousand rows with one bad line is a file with one bad line, not a failed import.

API

: i ANOM_IMPORT_MAX_BYTES 67108864

A file bigger than this is refused before it is parsed. Generous enough for years of minute-resolution history, small enough that a mistaken upload cannot exhaust memory.

: i ANOM_IMPORT_MAX_ROWS 1000000

: i ANOM_IMPORT_MAX_NOTES 5

How many bad rows are described rather than merely counted.

: ImportParse

: ImportParse {
    ( Vec Json ) rows
    i skipped
    ( Vec String ) notes  // why the first few were skipped
    String err  // non-empty ⇒ nothing was read at all
    String format  // what it turned out to be
}

@ import_parse_free ImportParse ip → v

@ import_sniff s text → String

Which of the three this body is. [ starts a JSON array; { starts either a lone object or the first line of JSONL, and which one is settled by whether a later line also starts an object.

@ import_parse s text s format → ImportParse

---

mcp.nu

anomaly/mcp.nu — the service as an MCP server, mounted at /mcp.

A language model talks to the same service the dashboard does, with the same credential, and gets exactly what that credential may do: the tool list is computed per caller (a viewer never sees delete_model), and every tool runs as an in-process HTTP request through the service's own router, so the API's authorisation gates are the only gates — there is no second rule book to drift.

What the model sees is shaped for a context window rather than for a chart: timestamps are ISO-8601 UTC, floats are rounded, a listing is a summary plus the newest rows, and every reply names what it left out.

POST /mcp JSON-RPC (Streamable HTTP transport) GET /.well-known/oauth-protected-resource[/mcp] where to get a token (RFC 9728)

Sign-in: the same OAuth (Entra) tokens the dashboard sends, or an API key in Authorization: Bearer / X-API-Key. With authorisation off (simple mode) every caller is the administrator, as everywhere else.

The server object is static (built once, no per-request state); the caller is a context Json the dispatch carries into gates and handlers.

API

: s ANOMALY_VERSION 0.13.0``

One version for the CLI banner and the MCP handshake.

: McpWiring

: McpWiring {
    Router router
    b has_router
    String public_url
    b has_server
    McpServer server
}

The router the tools call back into (a shallow copy of the service router, whose route table is complete by the time it is attached), the static server, and the public origin for the resource-metadata URLs.

: ~ i g_mcp_wiring 0

@ an_mcp_attach_router Router r → v

Called by anomaly_service_router once every route is registered. The copy shares the route vector, so the router must not grow afterwards.

@ an_mcp_set_public_url s url → v

[service] public_url — the origin clients reach the service at, when it sits behind a proxy that rewrites Host. Empty: derived per request from Host / X-Forwarded-*.

: ApiOut

: ApiOut {
    i status
    Json body  // the parsed body, or JSON null when it was not JSON
}

: FeatShare

: FeatShare {
    String name
    f share
    i n
}

Per-feature attribution totals across the flagged rows.

@ an_mcp_metadata_response HttpRequest req → HttpResponse

GET /.well-known/oauth-protected-resource[/mcp] — RFC 9728: which authorization server issues tokens for /mcp, and which scope to ask for. The same issuer and audience the dashboard uses. 404 in simple mode: there is no sign-in to point at.

@ an_mcp_handle HttpRequest req → HttpResponse

The /mcp endpoint. Called inside the service lock, like every handler.

---

csvdata.nu

anomaly/csvdata.nu — numeric CSV → row-major matrix (batch inputs).

Same forgiving shape as the iforest CLI parser: split lines, split on the delimiter, keep numeric fields; the first data row fixes the column count, later rows with a different numeric-field count are skipped. With has_header, the first line's names are captured (for batch reports that echo column values by name) instead of being parsed as data.

API

: AnomCsv

: AnomCsv {
    ( Vec f ) data
    i rows
    i cols
    ( Vec String ) headers
}

Parsed matrix: data is row-major rows*cols; headers is empty when the input had no header row.

@ anom_csv_free AnomCsv ds → v

@ anom_parse_csv s input s delim b has_header → AnomCsv

---

analyze.nu

src/analyze.nu — one-shot analysis of a file

POST /api/analyze takes a file and answers with its anomalies: the file becomes a throwaway model, trained as carefully as the data allows — every forest version, the 64-16-64 autoencoder over the forest- filtered points, every margin set to a 1 % alert rate — scanned once, and thrown away. What remains is the result file in the organisation's folder (src/orgfiles.nu) and a status record.

The work runs in a CHILD PROCESS, anomaly analyze-job <task dir>, not in the service: training holds the GPU singleton and the random state, both of which the live detectors use, and a file that takes the trainer down must not take the service with it. The service and the job meet only in the task directory:

<root>/orgs/<org>/tasks/<id>/params.json what to do (service → job) /input the file as posted /status.json queued | running | done | failed /store/ the temporary model, gone when done

The service watches status.json; the job writes it (atomically) at each transition, and the thread that ran the child marks it failed when the process died without saying so.

API

: f ANA_TARGET_RATE 0.01

: i ANA_MIN_ROWS 10

: i ANA_INLINE_MAX 10000 // anomalies returned in the response body; beyond it, a link

@ analyze_task_dir s org s id → String

@ analyze_status_read s dir → ?Json

@ analyze_status_write s dir Json st → b

@ _ana_jstr Json o s key → String

@ _ana_jint Json o s key i dflt → i

@ _ana_jbool Json o s key → b

@ analyze_state s dir → String

@ analyze_state_final s state → b

@ analyze_task_create s org Json params ( Vec u ) input → String

Create the task: a fresh id, its directory, the input and the parameters written, status queued. Returns the id ("" = could not create).

@ analyze_task_list s org → Json

Every task of the organisation, newest first: the status records, each with its id.

@ analyze_task_delete s org s id → b

: ~ s g_ana_exe ``

The service's own binary: the job is the same program in another process. The binary that runs jobs: this one, normally. A test binary is not anomaly — re-spawning it would run the whole suite again, in the same store — so tests point this at something harmless.

@ analyze_set_exe s exe → v

@ analyze_exe → String

@ _ana_mark_crashed s dir i code s detail → v

The child died without writing a final status: say so. A status the job wrote itself stands.

@ analyze_spawn s org s id → b

Start the job for a task: a thread runs the child process to its end and records a crash. The thread shares nothing with the service but the task directory. Returns F when no thread could be started.

@ analyze_run s dir → i

---

aegpu.nu

anomaly/aegpu.nu — GPU-accelerated autoencoder training (bit-exact).

The autoencoder trains through mlp_fit — minibatch Adam over a small dense net — and that training is where /train/autoencoder spends its time (~20 s for 20 k rows; the bulk-MSE pass after it is milliseconds). This module runs THE SAME training on the GPU when one is present, with the package's standing guarantee intact: backend choice can never change a result. The GPU path is bit-for-bit identical to mlp_fit, so a model trained on a CUDA machine equals the CPU-trained model exactly — same weights, same threshold, same verdicts. GPU is pure speed.

How bit-exactness is achieved (and pinned by tests/aegpu_parity_smoke.sh):

• Control flow stays on the host and MIRRORS mlp_train exactly: the same

seeded PRNG call order (init, the one full shuffle, the per-epoch train -region shuffles), the same validation split, minibatch bounds, early- stopping/patience/best-weights logic, and the same Adam scalar bookkeeping (lr_t via host pow/sqrt, passed to the kernel as data).

• The kernels reproduce the CPU's floating-point ROUNDING AND ORDER:

every accumulation chain uses explicit __dadd_rn/__dmul_rn/__dsub_rn (round-to-nearest, never fused — NVRTC's default fmad contraction would change results), dot products run serially in the CPU's k-order inside one thread, and per-weight gradient sums walk the batch rows in the CPU's row order. Division and sqrt are IEEE-correctly-rounded on both sides. Scalars that need libm pow are computed on the HOST and shipped to the kernel, so no transcendental runs on the device.

• Loss/criterion sums come back as per-element differences and are

folded on the host in the same flat order as the CPU loop.

Weights, Adam state and the data live on the device for the whole run; per minibatch the device does forward → backprop → gradient → Adam in four launches and only the output-layer diffs (batch×d f64s) come back.

Selection: engine cuda (score.nu's probe) → GPU; anything else — no device, ANOMALY_GPU=0, any mid-setup failure — falls back to the pure mlp_fit, which (being bit-identical) is a pace change, never a behaviour change. The gpu package's host-C++ backend is deliberately NOT used for training: native mlp_fit already is the optimized CPU path.

API

@ _aeg_eval_chunk → i

Rows per eval chunk (validation pass): bounds the acts buffer.

@ _aeg_kernel_src → s

── the kernels ─────────────────────────────────────────────────────── meta (i64): [0]=n_layers [1]=n_a [2]=d [3]=dout, then sizes(nl+1), w_off(nl), b_off(nl), a_off(nl+1). One block per row for fwd/bwd/diff (threads stride the neurons, __syncthreads between layers); one thread per parameter for grad/adam.

: AeGpu

: AeGpu {
    b ok
    Gpu g
    GpuKernel kfwd
    GpuKernel kbwd
    GpuKernel kgrad
    GpuKernel kadam
    GpuKernel kdiff
    GpuBuffer bmeta
    GpuBuffer bx
    GpuBuffer bidx
    GpuBuffer bw
    GpuBuffer bb
    GpuBuffer bmw
    GpuBuffer bvw
    GpuBuffer bmb
    GpuBuffer bvb
    GpuBuffer bgw
    GpuBuffer bgb
    GpuBuffer bacts
    GpuBuffer bdeltas
    GpuBuffer bodt  // train outdiff: bsz × dout
    GpuBuffer bode  // eval outdiff: eval_chunk × dout
}

── device context ──────────────────────────────────────────────────── Everything one aegpu_fit run holds on the device. All buffers are f64 except idx / meta (i64). X doubles as Y (autoencoder: target == input).

@ _aeg_free AeGpu cx → v

@ _aeg_meta Mlp m i d i dout → ( Vec i )

The meta buffer content for a freshly built net.

@ _aeg_open Mlp m i n i d i dout i bsz → AeGpu

Open the context: buffers + the five kernels, on score.nu's device singleton. Any failure → ok=F (caller falls back to mlp_fit).

@ _aeg_train AeGpu cx Mlp m ( Vec f ) X i n i d i dout MlpCfg cfg * u fail → MlpTrain

── the mirrored trainer ────────────────────────────────────────────── Exactly mlp_train, with the four per-minibatch compute steps and the validation forward on the device. Returns MlpTrain; the model's w/b, Adam state and t are updated in place, exactly as mlp_train leaves them. On ANY device error sets *fail and returns immediately (the caller discards everything and reruns on the CPU).

@ aegpu_fit ( Vec i ) sizes ( Vec f ) X i n i d MlpCfg cfg i restarts → MlpFit

── the public entry: mlp_fit, GPU when a CUDA device is present ───── Bit-identical to ( mlp_fit sizes X n d X d cfg restarts ) — the autoencoder case (target == input). Falls back to mlp_fit whenever the engine is not cuda or any device step fails.

---

prep.nu

anomaly/prep.nu — feature preprocessing, standardisation, model metadata.

This is milestone M1 of the anomaly package (see SPEC.md): the pure, I/O-free foundation everything else builds on.

• A model's metadata (*Meta) records what it has learned about the

shape of its input: which columns exist, their types (numeric / categorical / timestamp), the categories seen so far, the authoritative feature order, and the scaler parameters. It round-trips through JSON.

• anomaly_preprocess turns one raw JSON record into named numeric

features, updating the metadata as new columns / categories appear: numeric passthrough, categorical → deterministic one-hot (categories kept sorted), ISO-8601 timestamp → hour/day/month/weekday.

• anomaly_project pins a named feature set onto the model's frozen

feature order: missing features become 0, unknown extras are dropped. This is the feature-order stability rule that keeps one-hot columns aligned across retrains.

• Scaler is the StandardScaler analogue: per-feature zero-mean /

unit-variance, with zero-variance features passing through unscaled.

Column types are detected once, on first sight, then frozen in metadata: a value that parses as a number is numeric, else a value that parses as ISO-8601 is a timestamp, else it is categorical (mirrors the Python reference's float()/fromisoformat()/str fallback chain).

API

: i COL_NUMERIC 0

: i COL_CATEGORICAL 1

: i COL_TIMESTAMP 2

: i ANOM_MIN_POINTS 50

Reference defaults (carried over verbatim from the Python service).

: i ANOM_MAX_POINTS 150000

: i ANOM_SCHED_BELOW 50

: i ANOM_SCHED_AT_MAX 1000

: VerCfg

: VerCfg {
    String vname
    i window_min
    i window_pts
    i window_size  // sliding-window LENGTH in points (timevector; 0 = plain)
    i step_size  // sliding-window step during training (timevector)
    i n_estimators
    i max_samples
    f contamination
    f decision_margin
    b enabled
}

Config of one time-window model version. window_min filters training data to the last N minutes (0 = no time filter); window_pts caps it to the last N points (0 = no cap; used by timevector). contamination < 0 means "auto" (offset pinned at -0.5, the sklearn convention).

: Meta

: Meta {
    String name
    String created
    String alias  // human-readable nickname; empty = go by `name`
    ( Vec String ) cols
    ( Vec i ) kinds
    ( Vec ( Vec String ) ) cats
    ( Vec String ) feats
    ( Vec f ) sc_mean
    ( Vec f ) sc_std
    i sched_below
    i sched_at_max
    b sched_ae  // retrain the autoencoder whenever the forests retrain
    b count_clock  // points are numbered, not timed: see ANOM_TICK
    i n_seen
    i last_trained
    i max_points
    i score_epoch
    ( Vec VerCfg ) versions
}

Everything a model knows about its input shape. Heap-allocated (*Meta) so counters and handles can be updated through any reference. cols / kinds / cats are parallel: cats[i] is the sorted category list of column i (empty unless categorical). feats is the authoritative feature order once non-empty (snapshotted at each train by meta_refresh_feats); until then the model is "unfrozen" and the order is derived on demand.

: EncPoint

: EncPoint {
    ( Vec String ) names
    ( Vec f ) vals
}

One preprocessed record: parallel (feature name, value) pairs in encounter order. Project onto a feature order with anomaly_project.

: Scaler

: Scaler {
    ( Vec f ) mean
    ( Vec f ) inv_std
}

Per-feature standardisation: y = (x - mean) * inv_std. Zero-variance features store inv_std = 1 so they pass through centred but unscaled.

@ meta_default_versions → ( Vec VerCfg )

@ _an_vercfg_free VerCfg vc → v

: i ANOM_ALIAS_MAX 120

An alias is a label, so it is bounded by what a human will read rather than by anything structural. Long enough for a sentence, short enough that it cannot be used to bloat every metadata response.

: i ANOM_TICK 60

── The count clock ───────────────────────────────────────────────────

Data without a clock — a file of readings nobody dated, a sequence of measurements where only the order matters — is a model whose time is its point count. Such a model runs on the COUNT clock: every point is stamped one tick after the previous one, ticks are ANOM_TICK seconds apart, and nothing in it ever reads the wall clock. One tick is one minute on purpose: every version window is written in minutes, so on the count clock "180 minutes" reads as "the last 180 points", and every window, scan and calibration falls out of the same code unchanged. The number a point shows is its ordinal, timestamp ÷ ANOM_TICK.

@ meta_new s name s created → *Meta

@ meta_free * Meta m → v

@ meta_is_frozen * Meta m → b

A model is "frozen" once it has an authoritative feature order (set at first train). Before that, feature order is derived from the metadata.

@ meta_derived_feats * Meta m → ( Vec String )

The feature order implied by the current metadata: columns in first-seen order, each expanded canonically (categoricals over their sorted categories). Deterministic for a given metadata state. Owned result.

@ meta_refresh_feats * Meta m → v

Snapshot the derived feature order as authoritative (called at train time). From now on scoring projects onto exactly this vector.

@ enc_free EncPoint p → v

@ enc_find EncPoint p s name → i

Index of feature name in the encoded point, or -1.

@ anomaly_preprocess * Meta m Json raw → !EncPoint String

Encode one raw JSON record into named numeric features, updating the metadata as new columns / categories appear. The reserved key timestamp is the point's own clock and is never a feature. Numeric parse failure and bad timestamps are hard errors (owned message).

@ anomaly_preprocess_ro * Meta m Json raw → !EncPoint String

Read-only encode: never touches the metadata. Unknown columns are skipped, unseen categories one-hot to all-zeros — exactly what the frozen-feature projection would do with them anyway. For detect-only paths that must not mutate model state.

@ anomaly_project EncPoint p ( Vec String ) feats → ( Vec f )

Project an encoded point onto an authoritative feature order: missing features become 0, features not in feats are dropped. Owned result.

@ scaler_fit ( Vec f ) data i n_rows i n_cols → Scaler

Fit per-feature mean and 1/std over a row-major matrix (population variance, like sklearn's StandardScaler). Zero-variance features get inv_std = 1 so they centre but never divide by ~0.

@ scaler_apply Scaler sc ( Vec f ) point → v

Standardise one point in place: x → (x - mean) * inv_std.

@ scaler_apply_matrix Scaler sc ( Vec f ) data i n_rows i n_cols → v

Standardise a whole row-major matrix in place.

@ scaler_free Scaler sc → v

@ meta_set_scaler * Meta m Scaler sc → v

Persist a fitted scaler into metadata (stored as mean + std; zero variance is stored as std = 1, matching its inv_std = 1).

@ meta_scaler * Meta m → Scaler

Rebuild a usable Scaler from persisted metadata. Owned result.

@ _an_jarr_of_strs ( Vec String ) xs → Json

( Vec String ) → JSON array of strings.

@ meta_to_json * Meta m → Json

Serialise metadata to an owned Json object (fixed field order, so the same metadata always stringifies identically).

@ _an_jint Json o s key i dflt → i

Integer field of a JSON object, or dflt when absent/mistyped.

@ _an_vercfg_of_json s vname Json vo → VerCfg

One version config out of its JSON object.

@ meta_from_json Json j → ?*Meta

Parse metadata back from JSON. None on malformed shape (missing/mistyped required fields); the partially-built Meta is freed on failure.

@ meta_to_json_str * Meta m → String

Convenience: metadata → compact JSON text (owned).

@ meta_from_json_str s src → ?*Meta

Convenience: JSON text → metadata; None on parse or shape errors.

@ meta_find_version * Meta m s vname → i

Index of the version named vname, or -1.

@ meta_version_enabled * Meta m s vname b dflt → b

Is version vname enabled? dflt when there is no such version.

@ meta_version_margin * Meta m s vname f dflt → f

Version vname's decision margin; dflt when there is no such version. The live margin of a version always comes from the CURRENT metadata, not from its forest blob — so margin changes (fine-tune, a config PUT) take effect immediately, without a retrain. The blob's stored margin is only the fallback for versions no longer present in the metadata.

@ meta_bump_epoch * Meta m → v

Bump the scoring epoch: the token every cached verdict is stamped with. Anything that can change what a stored point scores — a retrain, a new autoencoder, a margin edit, a version toggled on or off, a reset — bumps it, and every cache entry carrying an older epoch is stale by construction. One counter beats trying to reason about which caches a given edit could have invalidated.

@ _an_vercfg_sane VerCfg vc → VerCfg

Clamp a config into the range the trainer can actually honour, so a hand-written JSON patch can never produce a version that fails to train (or trains something nonsensical). contamination < 0 means "auto".

@ _an_vercfg_patch VerCfg vc Json vo → VerCfg

Patch one version config from a JSON object. Every field is optional: what the object omits keeps its current value, so the dashboard can PUT {"enabled": false} without restating the whole config.

@ meta_apply_versions_json * Meta m Json vers b replace → i

Apply a versions JSON object (the shape meta_to_json emits) to the metadata. Each key names a version and its value is a PARTIAL config; a key naming no existing version ADDS one, with _an_vercfg_of_json's defaults filling the gaps. With replace, the resulting list is exactly the keys given — versions the object omits are dropped, which is how the advanced JSON editor deletes one. Returns the version count afterwards, or -1 when vers is not a JSON object.

---

main.nu

anomaly — streaming anomaly detection from the command line.

anomaly detect <model> key=val ... ingest one point → verdict JSON anomaly score <model> key=val ... score only (no state change) anomaly batch [-f FILE] [-H] score a CSV (index<TAB>score) anomaly train <model> force a retrain now anomaly train-ae <model> train the autoencoder version anomaly calibrate <model> [--last S] alert rate at the current margins, margin for each standard rate anomaly finetune <model> [--rate R] set margins so R of the window is [--last S|all|own] [-n] flagged (default 1 % of 24 h; own = each version its period) anomaly reset <model> drop data+forests, keep the name anomaly rm <model> delete the model entirely anomaly ls list models in the store anomaly info <model> dump model metadata (pretty JSON) anomaly serve [--addr HOST:PORT] run the HTTP/JSON service + dashboard [--webroot DIR] (dashboard HTML dir; auto-located)

Values in key=val are auto-typed: numbers are numeric features, ISO-8601 strings are timestamps, anything else is categorical (one-hot). Verdicts print as one JSON object per invocation. The store lives under --store DIR (default $ANOMALY_HOME, else ~/.anomaly).

The command line is assembled with the cli package: one Cli, a flag set, and a handler per subcommand — routing, typed flags with the $ANOMALY_HOME / $ANOMALY_WEBROOT fallbacks, --help / --version and exit codes are the framework's job.

API

@ pline s x → v

: s ANOM_DEFAULT_ADDR 127.0.0.1:8811``

The bind address a bare anomaly serve uses. Named so the flag's default and the "was --addr actually given" test cannot drift apart.

@ main → i

---

autoenc.nu

autoenc.nu — the autoencoder model version.

The reference recipe (model_training.py train_autoencoder_model), in pure NURL over the mlp package:

1. a TEMPORARY Isolation Forest (contamination 10 %) is trained on the

standardised ring and every point it flags is dropped — the autoencoder must learn only NORMAL behaviour, and unlabeled rings contain the very anomalies we want it to spot later;

1. the surviving rows are MinMax-scaled and an MLP autoencoder

(default 64-32-64, ReLU, Adam, early stopping — sklearn MLPRegressor semantics via mlp_fit with deterministic restarts) learns to reconstruct them;

1. the detection threshold is the 95th percentile (nearest-rank) of

the training reconstruction errors.

Detection scores a point as threshold − mse: negative ⇒ anomaly — the same decision_function orientation as the forest versions, so the service's "any version flags it" aggregation needs no special case.

The autoencoder is NEVER part of the automatic retrain schedule: it is trained explicitly (CLI anomaly train-ae, HTTP POST /train/autoencoder/<model>) and stays enabled until the model is reset — mirroring the reference, where a heavier, deliberately-trained version coexists with the self-training forests.

API

: AeModel

: AeModel {
    Mlp net
    MinMax mm
    ( Vec String ) feats  // feature order frozen at AE train time
    f threshold  // p95 of the training reconstruction errors
    i trained_on  // normal rows the net was fitted on
    i filtered  // rows the pre-filter dropped as anomalous
    f prefilter  // the pre-filter's contamination rate actually used
    i trained_at  // unix seconds (0 = unknown, pre-0.10 file)
    b trained
}

@ ae_empty → AeModel

A structurally-valid untrained placeholder (freeable, never scored).

@ ae_free AeModel ae → v

: AeTrainOut

: AeTrainOut {
    AeModel ae
    String err
}

Train from the RAW (unscaled) projected matrix raw (n×d, row-major) with its feature order feats (borrowed; copied into the result). hidden lists the hidden layer widths (empty → 64-32-64); contamination is the pre-filter rate (<=0 → 0.10); min_rows gates both the input and the post-filter survivor count. On failure the returned AeModel has trained=F and err (owned) says why.

@ ae_train_matrix ( Vec f ) raw i n i d ( Vec String ) feats ( Vec i ) hidden f contamination i min_rows → AeTrainOut

@ ae_hidden AeModel ae → ( Vec i )

The hidden layer widths the net was built with (sizes minus the input and output layers) — what a retrain needs to rebuild the same shape.

@ ae_mse AeModel ae ( Vec f ) raw_point → f

Reconstruction MSE of one RAW projected point (the AE's own feature order and MinMax are applied here — independent of the forests' standardising scaler).

: f ANOM_AE_MARGIN 0.05

── The decision margin, and why it is RELATIVE ───────────────────────

Every forest version's decision_margin is an absolute offset on a decision_function whose scale is fixed by construction: sklearn's convention puts normal points near 0 and anomalies below it, so 0.06 means the same thing for every model. The autoencoder's score is threshold − mse, and MSE has no such fixed scale — it is the mean squared error of MinMax-scaled features, which lands wherever the data puts it (1e-3 for one model, 2e-4 for another).

The Python reference (model_training.py) nevertheless applies its shared absolute-margin rule to the autoencoder too: the branch computes is_anomaly = mse > reconstruction_threshold, and thirty lines later, at the same indentation as the branch itself, is_anomaly = bool(score <= -decision_margin) overwrites it unconditionally — with the autoencoder default margin of 0.05. Against a threshold of ~5e-4 that demands a reconstruction error a HUNDRED times the p95 of the training errors, which mutes the one version that models the joint distribution. The reference ships that version disabled by default, so the muting was never noticed.

We keep the tunable knob and put it on the only scale that travels: decision_margin for the autoencoder is a FRACTION of the model's own reconstruction threshold. The stored default 0.05 now reads "flag at 5 % above the p95 training error" — the documented intent — instead of "flag at p95 + 0.05", and the same number means the same thing on every model. See SPEC §5.5.

@ anom_ae_margin AeModel ae f rel → f

The effective ABSOLUTE margin for rel, so the verdict rule stays the one every version shares: decision_function <= -margin ⇒ anomaly. Substituting ae_decision gives mse >= threshold * (1 + rel).

@ anom_ae_rel_margin AeModel ae f abs_margin → f

Turn an effective absolute margin back into the relative one stored in the metadata (the inverse of anom_ae_margin; 0 when untrained).

@ ae_feature_errors AeModel ae ( Vec f ) raw_point → ( Vec f )

Per-feature squared reconstruction error of one RAW projected point, in the AE's own feature order (length = ae.mm.n_cols). This is the attribution the forests cannot give: the autoencoder's error is the amount by which each feature failed to be predictable from the others, so the largest entries name the RELATIONSHIP that broke, not merely the value that was extreme. ae_mse is the mean of this vector; it keeps its own loop because it is on the per-point scoring path and this one allocates.

@ ae_reconstruct AeModel ae ( Vec f ) raw_point → ( Vec f )

The autoencoder's reconstruction of one RAW projected point, back in raw units (the MinMax undone), in the AE's feature order. For a flagged point this is what each feature "should" have been given the others — the value the broken relationship expected. A feature whose training range was a single value reconstructs to that value.

@ ae_decision AeModel ae ( Vec f ) raw_point → f

decision_function orientation: threshold − mse (negative ⇒ anomaly).

@ ae_to_json_str AeModel ae → String

@ ae_from_json_str s src → ?AeModel

@ ae_dummy_net → Mlp

Fallback constructors for the load path's option unwrap (never used on the success path; both arms above guarantee have_net/have_mm).

@ ae_dummy_mm → MinMax

---

authz.nu

anomaly/authz.nu — who is asking, and which models they may touch.

The service was single-user by construction: every route reached every model in one flat store. This file adds the three things that turns into a shared service — an identity, a tenant, and an owner — without moving a single stored model.

identity An OIDC bearer token, verified by the oauth package against the provider's own JWKS. The token names a subject (sub) and, on Entra, a tenant (tid). tenant One SQLite database per organisation, at <store>/orgs/<org>.db. The org is implicit in the file, so no query in here carries an org column and no query can forget one. owner A row in that database binding a model name to a subject. A model with no row is UNOWNED: legacy data that predates authentication, visible to admins so somebody can claim it, never silently absorbed into whoever logged in first.

Three roles. admin manages the whole organisation's models and its users and keys; viewer reads every one of its models and may change none of them — except the scratch models named llm_…, which every member may build and destroy (az_is_scratch_model); ingest is for machines that send points (keys only, never a person). A model belongs to the organisation, so every member SEES the same set; the roles differ in what they may DO. The first subject to authenticate from an organisation becomes its admin — there is nobody else who could have granted it, and an org whose only user is a viewer would be permanently unadministrable.

API keys are for machines that cannot do an interactive login (the Node-RED flows feeding these models). A key carries the identity of the user who created it and a role of its own, admin or ingest. Keys are stored as a SHA-256 of the secret; the plaintext exists once, in the response that creates it.

EVERYTHING HERE IS OFF BY DEFAULT. With ANOMALY_AUTH unset the resolver hands every request an authenticated admin principal and the service behaves exactly as it did before this file existed. That is deliberate: a deployment upgrades the binary first and turns on authentication when its identity provider is configured, not at the same instant.

API

: ~ s g_az_root .``

: i AZ_MODE_SIMPLE 0

Two modes, and they are different products.

simple No authentication at all. Anyone who opens the page sees every model, and what the API collects lands in one public organisation. This is what the service was before sign-in existed, kept as a mode rather than as a fallback so that a deployment which wants it says so. oidc Signed in, multi-tenant. A model belongs to an ORGANISATION, and nothing is created or collected without a credential that names one.

: i AZ_MODE_OIDC 1

: s AZ_PUBLIC_ORG public``

The organisation everything belongs to in simple mode. A real organisation with a real database, so the two modes share one shape and a deployment can turn sign-in on later without moving data.

: ~ i g_az_mode 0 // AZ_MODE_SIMPLE; a global initialiser must be a literal

: ~ b g_az_open_ingest F

: AzStrings

: AzStrings {
    String s_issuer
    String s_client_id
    String s_audience
    String s_allowed
    String s_last_err  // why the last token verification failed
    String s_owner  // the owner tenant: config-only, the trust anchor
}

The configured strings, owned. A global can hold a s (a borrowed char*) but not a String, so the Strings behind these live in one heap block whose address is the global; reconfiguring frees the previous block. The obvious alternative — hand the globals ( string_data owned ) and never free the owned String — is a leak that a long-running service would never notice and a test that reconfigures would report every time.

: ~ i g_az_strs 0

@ g_az_issuer → s

@ g_az_client_id → s

@ g_az_audience → s

@ g_az_allowed → s

@ g_az_owner → s

The OWNER TENANT: the organisation whose admins administer the service itself — approving other tenants, managing any organisation's users. It is set in the configuration file and nowhere else. A tenant that could grant itself that from the dashboard would not be an anchor.

@ anomaly_authz_set_owner_tenant s tid → v

@ anomaly_authz_last_error → s

Why the last presented token was refused. A 401 with no reason is what turns a one-line configuration mistake — the wrong audience, a clock an hour out, a tenant nobody listed — into an afternoon. The service is single-threaded, so one slot is the whole story.

: ~ i g_az_prov_addr 0

The identity provider, discovered lazily and then reused: it owns the JWKS cache, and re-fetching a key set per request would turn every authenticated call into two network round trips. The service runs single-threaded (http_app_listen with no worker pool), which is the condition *OidcProvider documents for going unlocked.

Held as an address because a global cannot carry an option of a pointer; 0 means "not discovered yet". Discovery happens once and the provider then lives for the process, so nothing here frees it.

: ~ b g_az_multi F

── Multi-tenant ──────────────────────────────────────────────────────

A single-tenant application has one issuer, and a token either carries it or is refused. A MULTI-tenant one does not: every organisation signs its users' tokens with its own issuer, and the provider says so — Entra's discovery document at the multi-tenant authority literally publishes

"issuer": "https://login.microsoftonline.com/{tenantid}/v2.0"

which is a template, not a URL. So there is nothing to pin, and the oauth package's discovery refuses it: it cross-checks the document's own issuer against the one asked for, correctly, and a template never matches.

What is checked instead is what the provider itself documents: the token's iss must be that template with the token's OWN tid substituted. Both claims are inside the signature, so an attacker cannot move a token between tenants; what the check enforces is that a token claiming tenant X was issued by tenant X's issuer, which is the part a fixed string would have got wrong.

The trade is that ANY organisation can then sign in and have an organisation created for it. allowed_tenants is the list that says which may; empty means any, which is what "multi-tenant" asks for and should be a deliberate answer rather than a default nobody saw.

: ~ s g_az_iss_tmpl ``

@ anomaly_authz_set_root s root → v

@ anomaly_authz_configure b on b open_ingest s issuer s client_id s audience → v

on gates the whole thing; open_ingest keeps /detect and /detect_only reachable without credentials while a fleet of already-deployed data producers is migrated onto keys. It is a migration setting: with it on, anyone who can reach the port can write points into any model.

@ anomaly_authz_configure_tenancy b multi s allowed → v

Multi-tenant acceptance. allowed is a comma-separated tenant list; empty admits every organisation the provider will sign for.

@ anomaly_authz_multi_tenant → b

@ anomaly_authz_allowed_tenants → s

@ anomaly_authz_apply AnomalyConfig cfg → b

Resolve the settings from the configuration file and the environment, in that order — the file is the persistent baseline, the environment is what one run overrides. (A command-line flag beats both, but nothing here reads argv: main.nu owns that layer.) Returns T when authentication ended up enabled.

[auth] enabled / issuer / client_id / audience / open_ingest ANOMALY_AUTH ANOMALY_OIDC_ISSUER ANOMALY_OIDC_CLIENT_ID ANOMALY_OIDC_AUDIENCE ANOMALY_OPEN_INGEST

audience defaults to api://<client_id> — the app-id URI an OIDC provider hands out for an API the client registered for itself, which is what nearly every deployment wants and none should have to restate.

@ anomaly_authz_requested AnomalyConfig cfg → b

True when authentication was ASKED for — by file or environment — so a caller can tell "off because nobody asked" from "off because it was asked for without an issuer or a client id".

@ anomaly_authz_enabled → b

@ anomaly_authz_mode → i

@ anomaly_authz_simple → b

@ anomaly_authz_owner_tenant → s

@ anomaly_authz_open_ingest → b

@ anomaly_authz_issuer → s

@ anomaly_authz_client_id → s

@ anomaly_authz_audience → s

: s AZ_ROLE_ADMIN admin``

: s AZ_ROLE_VIEWER viewer``

: s AZ_ROLE_INGEST ingest``

A key is a credential, not a person, so it does not carry a person's role. A viewer is somebody who reads; a machine that reads is pointless and a machine that can delete a model it feeds is a hazard. The one thing a producer needs is to send points, so that is its own capability.

: Principal

: Principal {
    b authed
    b via_key  // arrived with an API key rather than a browser token
    String org
    String sub
    String email
    String pname
    String role
    String key_id  // empty unless via_key
}

@ principal_free Principal p → v

@ principal_anon → Principal

@ principal_public_admin → Principal

The principal every request gets in simple mode: an admin of the shared public organisation. A real organisation with a real database, so the two modes share one shape and turning sign-in on later does not have to move anything.

@ principal_local_admin → Principal

@ principal_is_admin Principal p → b

@ principal_is_owner_admin Principal p → b

An admin of the owner tenant: the one principal that reaches across organisations. Everything it can do, an ordinary admin can do inside its own organisation and nowhere else.

@ principal_may_ingest Principal p → b

May this credential send points to a model the organisation already has?

An admin may, because an admin may do anything here. A KEY may if it was issued to — that is what ingest names, and it is the whole point of having a separate capability: a production key should be able to feed a model without being able to delete it.

A viewer may not. A viewer reads what the models have collected and decided; sending points is not reading.

@ principal_json Principal p → Json

@ az_home_org → String

@ az_is_home_org s org → b

@ az_model_in_public s name → b

Is name held by the public organisation — the bucket for points that arrived without a credential naming an owner?

@ az_model_release_public s name → b

Release a model from the public organisation, so it can be adopted into a real one. Ownerless data is not the public organisation's property; it is data nobody has claimed yet, and public is where it waits.

@ az_db_open s org → !Database SqliteErr

Open (creating if absent) the organisation's database with its schema applied. The caller owns the handle; it closes at the caller's scope end.

: s AZ_TENANT_PENDING pending``

: s AZ_TENANT_ALLOWED allowed``

: s AZ_TENANT_BLOCKED blocked``

@ az_root_open → !Database SqliteErr

@ az_tenant_state Database db s tid → String

The recorded state of tid, or "" when it has never been seen.

@ az_tenant_note Database db s tid s label i now → String

Record that tid knocked. First time: pending. Afterwards the state is whatever was decided, and a repeat visit must not undo a decision.

@ az_tenant_set_state Database db s tid s state s by i now → b

@ az_tenant_forget Database db s tid → v

@ az_tenants_json Database db → Json

@ az_tenant_admitted s tid i now → b

May this tenant use the service? The owner tenant always may — it is the anchor the rest is decided from, and locking it out would leave nobody who could unlock anything. Everything else is a recorded decision, and a tenant nobody has decided on is recorded as pending and refused.

@ az_seed_allowed s csv i now → v

Seed the registry from the configuration file. Only ever ADDS: a deployment that lists tenants in its config gets them admitted without a dashboard, and a decision made in the dashboard is never undone by a restart.

@ az_user_count Database db → i

@ az_user_role Database db s sub → String

The role recorded for sub, or "" when the organisation has never seen them.

@ az_user_touch Database db s sub s email s name i now → String

Record that sub was here, and return the role they hold. The FIRST subject an organisation ever sees becomes its admin: there is nobody else who could grant it, and an organisation whose every user is a viewer can never appoint one.

@ az_user_set_role Database db s sub s role → b

@ az_admin_count Database db → i

@ az_users_json Database db → Json

@ az_model_owner Database db s name → String

The owner of name, or "" when the model has no row — UNOWNED, which is what every model created before this file existed looks like.

@ az_model_claim Database db s name s sub i now b force → b

Claim name for sub. force reassigns a model that already has an owner (admin territory); without it an owned model is left alone and the call reports F.

@ az_model_forget Database db s name → v

@ az_org_model_names Database db → ( Vec String )

Every model name this organisation claims, owned by anyone in it. This is the admin's whole world: the store is flat and global, so "every model" is every model of every ORGANISATION, and an admin of one tenant must never be shown another's.

@ az_all_owned_json Database db → Json

Every model the organisation has a row for, owned by anyone. An admin listing needs this only to know which stored models are still unclaimed.

@ az_may_see Database db Principal p s name → b

May this principal see the model at all?

A model belongs to an ORGANISATION, not to a person. Everyone in the organisation sees the same models; what differs is what they may do to them. That is the shape a shared service actually has — a colleague leaving must not take a production model with them — and it is why models has no owner column any more, only the database it sits in.

@ az_model_in_org Database db s name → b

Does this organisation hold this model?

Row EXISTENCE, not a non-empty creator: the two are different questions and conflating them means forgetting a person also forgets which organisation their models belonged to. owner_sub survives only as an audit trail of who first sent a point, and is blanked when they leave.

@ az_may_write Database db Principal p s name → b

May this principal CHANGE it? Only an admin. A viewer sees the organisation's models and the data they have collected — that is what a viewer is for — but retraining rewrites forests, a margin edit changes every verdict, and a reset destroys history. None of that is viewing.

The one exception is the organisation's SCRATCH namespace: a model named llm_… is an experiment, and an agent — or the viewer driving it — may build, tune and discard its own experiments without an administrator's signature. Production models keep the full rule. This is the law both the HTTP API and the MCP surface enforce, so the two never disagree.

: s AZ_LLM_PREFIX llm_``

Models whose name starts with this prefix are scratch models: every member of the organisation may create, retrain, tune and delete them.

@ az_is_scratch_model s name → b

Is this a scratch model by name? The bare prefix is not a name.

@ az_user_delete Database db s sub → b

@ az_org_models_before_delete Database db → ( Vec String )

Every model name this organisation holds, so a caller deleting the organisation knows what to remove from the store.

@ az_org_drop s org → b

Delete the organisation's database outright. The models themselves live in the shared store and are the caller's to remove — this only forgets who they belonged to, and doing both in one place would put a model deletion inside a function whose name says "database".

: s AZ_KEY_PREFIX anok_``

: KeyIssue

: KeyIssue {
    String key_id
    String secret  // the plaintext, which exists only here and in the response
}

@ key_issue_free KeyIssue k → v

@ az_key_create Database db s sub s label s role i now → KeyIssue

: String token ( string_from AZ_KEY_PREFIX )

The token the caller presents: prefix, id, secret.

@ az_key_revoke Database db s id i now → b

Revoke one of the organisation's keys. No owner check: the key is the organisation's, and whoever happened to press the button that created it has no more claim on it than any other admin.

@ az_keys_json Database db → Json

The keys sub may see: their own, or every one in the organisation when all is set (an admin listing). Secrets are never in here. The organisation's keys. There is no per-person view of them, because there is no per-person ownership: a key is the ORGANISATION's credential, and only its admins have any business seeing that one exists.

: KeyParts { b ok String kp_id String kp_secret }

Split "anok_<id>_<secret>" into its two halves. F when the token is not of that shape, which is also how the resolver tells an API key from a JWT.

@ key_parts_free KeyParts k → v

@ az_key_principal Database db s org KeyParts kp i now → Principal

Resolve a presented key against one organisation's database. The principal comes back unauthenticated when the id is unknown, the secret does not match, the key is revoked, or its owner has since been removed.

@ _az_org_ids → ( Vec String )

Every organisation database in the store, by org id. An API key names no organisation — the id alone would have to be globally unique to do that without a second index — so a presented key is tried against each. There is one database per tenant and a key arrives a few times a minute, so the scan is cheaper than a second source of truth that could disagree.

@ authz_principal_at HttpRequest req i now → Principal

The one entry point the service uses: turn a request into a principal. With authentication off this is an admin of the local organisation, which is what keeps every existing deployment working unchanged.

@ authz_principal HttpRequest req → Principal

---

score.nu

anomaly/score.nu — bulk scoring, version training, and the stateless batch path (milestones M2 + M7's GPU acceleration).

Bulk scoring routes through the gpu package when profitable:

• gpu_open picks CUDA when a device is present, else the gpu

package's CPU backend (the same kernel compiled by the host C++ compiler and run under OpenMP). If neither is available — no GPU, no C++ compiler — scoring silently stays on the pure-NURL loop, so a bare machine behaves exactly like pre-GPU builds.

• Output is BIT-IDENTICAL across all three paths, by construction: the

kernel only walks the trees and accumulates f64 path lengths in the same order as the pure loop, using per-leaf c(size) values precomputed on the host by the very same iforest_avg_path; the nonlinear finish (2^(-avg/c), the offset subtraction) runs in NURL either way. IEEE-754 f64 adds in a fixed order give the same bits on every backend — so backend choice can never change a verdict.

ANOMALY_GPU=0 disables the accelerator outright; NURL_GPU=cpu (a gpu package knob) forces its CPU backend on a CUDA machine. Batches under ANOM_GPU_MIN_ROWS rows skip the accelerator — upload latency would beat the win, and bit-equality makes the switch invisible.

API

: i ANOM_GPU_MIN_ROWS 128

Below this many rows the pure loop wins on latency. Bit-equality of the paths makes the threshold unobservable in results.

: BatchReport

: BatchReport {
    i total_rows
    i anomaly_count
    f anomaly_percentage
    ( Vec i ) anomaly_indices
    b has_anomalies
    ( Vec f ) scores
}

Batch verdict over a matrix (see SPEC §6): anomaly ⇔ predict == -1, i.e. decision_function < 0 (margin 0), matching the reference batch path.

: ~ i g_ag_state 0

: ~ i g_ag_kit 0

@ anom_gpu_kit → *GpuKit

Public view of the kit for sibling modules (aegpu.nu shares the device — __-prefixed functions are file-scoped). Only valid after anom_gpu_engine reported cuda or cpu.

@ anom_gpu_close → v

Release the device + kernel cache (tests call this so leak checkers see a closed shop; a long-running service just keeps the singleton).

@ anom_gpu_engine → s

Which engine bulk scoring would use right now: cuda, cpu (the gpu package's host backend) or none (pure NURL loop).

@ anom_scores_cpu VerModel vm ( Vec f ) scaled i n_rows i n_cols → ( Vec f )

Pure-NURL reference path: iforest_score per row.

@ anom_scores_gpu VerModel vm ( Vec f ) scaled i n_rows i n_cols → ?( Vec f )

Accelerated path. None when the accelerator is unavailable or any step fails (caller falls back; failures after a good probe are reported).

@ anom_scores VerModel vm ( Vec f ) scaled i n_rows i n_cols → ( Vec f )

Raw iforest scores for every row of an already-standardised matrix, through the best available engine. Bit-identical on all engines.

@ anom_decisions VerModel vm ( Vec f ) scaled i n_rows i n_cols → ( Vec f )

decision_function for every row: -score - offset, same expression as anom_decision.

@ _an_percentile ( Vec f ) sorted f q → f

q in [0,1] over an ASCENDING-sorted vector.

@ anom_train_version ( Vec f ) scaled i n_rows i n_cols VerCfg cfg → VerModel

Train one version's forest over an ALREADY-STANDARDISED row-major matrix. cfg.max_samples is clamped to the row count; contamination < 0 = "auto" (offset pinned at -0.5), else offset_ is the 100*c percentile of the training set's score_samples — so predict == -1 flags ~c of training.

@ anomaly_batch ( Vec f ) data i n_rows i n_cols VerCfg cfg → BatchReport

Fit a scaler over the raw matrix, train one version on the standardised copy, then score every row. Anomaly ⇔ decision_function <= -cfg.margin (margin 0 reproduces the reference's predict == -1 batch rule).

@ anomaly_report_free BatchReport rep → v

---

store.nu

anomaly/store.nu — persistence (milestone M3).

On-disk layout, one directory per model under the store root:

<root>/<name>/ metadata.json M1 metadata (feature order, scaler, versions…) data.jsonl raw ingested points, one JSON record per line (raw records — not projected vectors — so a retrain can pick up new categories/columns) version_<v>.forest one binary forest blob per trained version autoencoder.json the trained autoencoder, if any scores.bin cached per-point verdicts (epoch-stamped)

The forest blob ("ANOMFOR1") is a little-endian dump of the iforest node arena plus the version's decision offset/margin. Loading re-validates every structural invariant (lengths, index ranges, node-count caps), so a corrupt or truncated blob comes back as None — never undefined behaviour.

Writes are atomic: serialise to <file>.tmp, then rename(2) over the final name, so a crash mid-write can't leave a half-written model.

API

: i ANOM_BLOB_MAX_NODES 200000000

Refuse to load blobs claiming more than this many arena nodes / trees / name bytes — bounds untrusted counts before any allocation.

: i ANOM_BLOB_MAX_TREES 100000

: i ANOM_BLOB_MAX_NAME 4096

: BlobRd

: BlobRd {
    ( Vec u ) buf
    i pos
    b ok
}

@ vermodel_to_bytes VerModel vm → ( Vec u )

Serialise one trained version (forest + offset + margin) to owned bytes.

@ vermodel_from_bytes ( Vec u ) buf → ?VerModel

Parse a forest blob. None on any structural violation: bad magic, short buffer, absurd counts, mismatched array lengths, out-of-range indices.

: Store

: Store {
    String root
}

@ store_open s root → Store

@ store_free Store st → v

@ store_exists Store st s name → b

A model exists iff its metadata file does.

@ store_list Store st → ( Vec String )

Sorted names of every model in the store (dirs with a metadata.json).

@ store_save_meta Store st s name * Meta m → b

Persist metadata (creates the model directory as needed).

@ store_load_meta Store st s name → ?*Meta

@ store_save_forest Store st s name VerModel vm → b

Persist one trained version's forest blob.

@ store_save_ae Store st s name AeModel ae → b

Persist / load the autoencoder version (one JSON per model).

@ store_load_ae Store st s name → ?AeModel

@ store_load_forest Store st s name s vname → ?VerModel

Load one version's forest; None if absent or corrupt. The blob's embedded version name must match the requested one — a renamed or content-tampered file is treated as corrupt, not trusted.

@ store_delete_forest Store st s name s vname → v

Remove a trained version's blob (used by reset). Missing file is fine.

@ store_delete Store st s name → b

Delete a model entirely (directory and everything in it).

: i ANOM_SC_UNSCORED 0

── Scored-verdict cache (scores.bin) ─────────────────────────────────

Re-scoring a stored ring is pure recomputation: the same point against the same forests yields the same verdict every time. The dashboard's anomaly scan used to pay for that recomputation on every visit, one HTTP round trip AND one full model load per point, which is what made a 5000-point scan a minutes-long progress bar.

The cache is a ring-aligned array of verdicts stamped with the model's score_epoch (prep.nu). Anything that can change a verdict — a retrain, a new autoencoder, a margin edit, a version toggled, a reset — bumps the epoch, and the whole cache is stale by construction; there is no per-entry invalidation rule to get wrong.

Alignment survives ring eviction because rows are keyed on the LIFETIME point counter, not the ring index: base_seen is the lifetime index of row 0, so a ring row j of a ring of length L at counter S sits at cache index S - L + j - base_seen. Rows outside [0, nrows) are simply misses.

"ANOMSCR1" | u64 epoch | u64 base_seen | u64 nver | nver × (u64 len, bytes) | u64 nrows | nrows × (f64 score, u64 state, u64 present, u64 flagged)

state is 0 for a row never scored under this epoch, 1 for a scored verdict and 2 for "the model was not ready for this point". present and flagged are bitmasks over vnames, so a version that produced no verdict (a timevector window longer than the ring prefix) stays distinguishable from one that produced a clean verdict.

: i ANOM_SC_SCORED 1

: i ANOM_SC_NOT_READY 2

: i ANOM_SC_MAX_ROWS 100000000

Refuse a cache claiming more rows/versions than a model could hold.

: i ANOM_SC_MAX_VERS 64

: ScoreCache

: ScoreCache {
    i epoch
    i base_seen
    ( Vec String ) vnames
    ( Vec i ) state
    ( Vec f ) score
    ( Vec i ) present
    ( Vec i ) flagged
}

@ scorecache_new i epoch i base_seen → ScoreCache

@ scorecache_free ScoreCache c → v

@ scorecache_rows ScoreCache c → i

@ scorecache_resize ScoreCache c i n → v

Grow to n rows, every new row unscored.

@ scorecache_set ScoreCache c i at i state f score i present i flagged → v

@ scorecache_vnames_match ScoreCache c ( Vec String ) live → b

Do the cached version names still match the live ones, in order?

@ store_save_scores Store st s name ScoreCache c → b

@ store_load_scores Store st s name → ?ScoreCache

Load the cache; None when absent, truncated or structurally impossible. A rejected cache costs a rescan, never a wrong verdict.

@ store_delete_scores Store st s name → v

@ store_append_point Store st s name s line → b

Append one line (the record's compact JSON, no newline).

@ store_write_points Store st s name ( Vec String ) lines → b

Rewrite the whole log (ring eviction / reset).

@ store_load_points Store st s name → ( Vec String )

Load the log as owned lines (empty vec if the file doesn't exist).