vindex.nu

vindex.nu — a vector index: search the k nearest of n d-dim vectors by cosine or L2. Two builders share one search surface:

EXACT — brute force over every vector (the ground truth). IVF-flat — a k-means coarse quantiser (nlist centroids) with inverted lists; a query scores only the nprobe nearest clusters, trading recall for speed on a knob.

Cosine precomputes each vector's L2 norm, so a candidate costs one dot product. Scores are "smaller = nearer" (cosine: 1 − cossim; L2: squared distance), so the same top-k machinery serves both. The index serialises to one .vix byte blob and loads back to identical results.

( vx_build_exact data n dim metric ) → VIndex ( vx_build_ivf data n dim metric nlist niter seed ) → VIndex ( vx_search idx q k nprobe out_ids out_dists ) → i (results found) ( vx_free idx ) → v early release (optional) ( vx_save idx ) → ( Vec u ) ( vx_load bytes ) → !VIndex String

metric: VX_COSINE (0) or VX_L2 (1).

A builder takes ownership of data (sink). A VIndex is a handle: every copy is the same index, and the last owner releases it.

API

: i VX_COSINE 0

: i VX_L2 1

: VIndexImpl

: VIndexImpl {
    ( Vec f ) data  // n·dim, row-major
    ( Vec f ) norm  // n L2 norms (for cosine)
    i n
    i dim
    i metric
    i nlist  // 0 = exact
    ( Vec f ) cent  // nlist·dim centroids
    ( Vec f ) cnorm  // nlist norms
    ( Vec i ) list_off  // nlist+1 CSR offsets into members
    ( Vec i ) members  // n vector ids grouped by cluster
}

: VIndex { s ctl }

@ VIndex_share VIndex h → VIndex

@ VIndex_drop sink VIndex h → v

@ vx_build_exact sink ( Vec f ) data i n i dim i metric → VIndex

@ vx_build_ivf sink ( Vec f ) data i n i dim i metric i nlist i niter i seed → VIndex

@ vx_search VIndex idx__h ( Vec f ) q i k i nprobe ( Vec i ) out_ids ( Vec f ) out_dists → i

Fill out_ids/out_dists (cleared first) with the k nearest of query q. nprobe is ignored for an exact index. Returns the number found.

@ vx_n VIndex idx__h → i

@ vx_dim VIndex idx__h → i

@ vx_nlist VIndex idx__h → i

@ vx_free sink VIndex idx → v

Let go of idx now rather than at the end of its owner's scope.

@ vx_save VIndex idx__h → ( Vec u )

@ vx_load ( Vec u ) b → !VIndex String