tensor.nu

tensor/tensor.nu — an n-dimensional array over gpukit.

The reusable ndarray the ML packages want: onnx keeps its tensors welded to the graph runtime, and gpukit is a marshalling floor nobody wants to program against. Tensor is the shared layer — shape, broadcasting, matmul and reductions — that onnx, anomaly and a future training package can all sit on.

Storage is row-major contiguous f64 (NURL's native float). dtype records the logical type: an F64 tensor is exact f64; an F32 tensor keeps its values on the f32 grid (results are rounded to f32 after each op), so it behaves like float32 for interop while computing in f64. Bytes in/out (from_f32 / to_f32) bridge to onnx weights and files.

API

: i TE_F64 0

: i TE_F32 1

: Tensor

: Tensor {
    i dtype
    ( Vec i ) shape
    ( Vec f ) data
}

@ _t_prod ( Vec i ) shape → i

@ _ti ( Vec i ) v i k → i

@ _tf ( Vec f ) v i k → f

@ _t_absent Tensor t → b

A tensor with a SHAPE but no values — a constant whose data arrives later (grad's grad_const_lazy, so a model's frozen weights never all sit in host RAM at once). Every accessor above is bounds-checked, so reading one just yields zeros; what it must never reach is a kernel that uploads data sized by the SHAPE. That upload would read past a zero-length buffer, and on CUDA the error is sticky: the next allocation fails instead, pointing at the wrong place entirely.

@ _t_strides ( Vec i ) shape → ( Vec i )

Row-major strides for a shape.

@ _ivec_t i n i fill → ( Vec i )

@ _fvec_t i n f fill → ( Vec f )

@ _shape_copy ( Vec i ) s → ( Vec i )

@ _t_round32 f x → f

Round an f64 to the nearest float32-representable value.

@ tensor_of i dtype ( Vec i ) shape ( Vec f ) data → Tensor

Adopt a data vector (moved in) under a shape. nelem must equal prod(shape).

@ tensor_free sink Tensor t → v

Let go of t now rather than at the end of its owner's scope (a Tensor is an owning struct: its shape and data go with it).

@ tensor_full i dtype ( Vec i ) shape f v → Tensor

@ tensor_zeros i dtype ( Vec i ) shape → Tensor

@ tensor_ones i dtype ( Vec i ) shape → Tensor

@ tensor_from_data i dtype ( Vec i ) shape ( Vec f ) data → Tensor

From an explicit row-major f64 buffer (data is copied and rounded to the dtype grid; shape is adopted — the tensor takes ownership of it).

@ tensor_arange i dtype i n → Tensor

1-D [0, 1, …, n-1].

@ tensor_ndim Tensor t → i

@ tensor_size Tensor t → i

@ tensor_dtype Tensor t → i

@ tensor_dim Tensor t i ax → i

@ tensor_shape Tensor t → ( Vec i )

@ tensor_data Tensor t → ( Vec f )

@ tensor_flat Tensor t i k → f

Flat element read/write.

@ tensor_set_flat Tensor t i k f v → v

@ tensor_reshape Tensor t sink ( Vec i ) shape → ?Tensor

The same data under a new shape (must have equal nelem). Data is copied so the result is independent; shape is consumed — adopted by the result, or dropped on a size mismatch.

@ tensor_clone Tensor t → Tensor

Clone a tensor (independent copy).

@ tensor_astype Tensor t i dtype → Tensor

---

dev.nu

tensor/dev.nu — DEVICE-RESIDENT tensors (M3).

A DTensor is a Tensor whose data lives in GPU memory (a gpukit GkBuf): ops chain on the device with no host roundtrips, which is what a NN forward pass needs (upload the weights once, stream activations through). Residency is EXPLICIT — tensor_to_device / dtensor_to_host move data, ops never sync behind your back.

Numerics: a TE_F32 DTensor computes IN float32 on the device — true float32 semantics (accumulation included), matching numpy float32 and onnxruntime. That differs from the HOST TE_F32 Tensor, which computes in f64 and rounds each result to the f32 grid: elementwise results agree exactly, but reductions/matmul accumulate differently (f32 vs f64 sums). TE_F64 DTensors are bit-identical to host ops wherever the kernel accumulates sequentially (elementwise, matmul).

API

: DTensor

: DTensor {
    i dtype
    ( Vec i ) shape
    GkBuf buf
}

@ dtensor_ok DTensor d → b

@ dtensor_free sink DTensor d → v

Let go of d now rather than at the end of its owner's scope: a DTensor owns its shape and its device block (a GkBuf), and both go with its last owner.

@ dtensor_ndim DTensor d → i

@ dtensor_size DTensor d → i

@ dtensor_dtype DTensor d → i

@ dtensor_dim DTensor d i ax → i

@ tensor_to_device GpuKit kit Tensor t → DTensor

@ dtensor_to_host GpuKit kit DTensor d → Tensor

@ dtensor_add GpuKit kit DTensor a DTensor b → ?DTensor

@ dtensor_sub GpuKit kit DTensor a DTensor b → ?DTensor

@ dtensor_mul GpuKit kit DTensor a DTensor b → ?DTensor

@ dtensor_div GpuKit kit DTensor a DTensor b → ?DTensor

@ dtensor_adds GpuKit kit DTensor a f v → ?DTensor

@ dtensor_subs GpuKit kit DTensor a f v → ?DTensor

@ dtensor_muls GpuKit kit DTensor a f v → ?DTensor

@ dtensor_divs GpuKit kit DTensor a f v → ?DTensor

@ dtensor_relu GpuKit kit DTensor a → ?DTensor

@ dtensor_sigmoid GpuKit kit DTensor a → ?DTensor

@ dtensor_exp GpuKit kit DTensor a → ?DTensor

@ dtensor_tanh GpuKit kit DTensor a → ?DTensor

@ dtensor_sqrt GpuKit kit DTensor a → ?DTensor

@ dtensor_log GpuKit kit DTensor a → ?DTensor

@ dtensor_matmul GpuKit kit DTensor a DTensor b → ?DTensor

@ dtensor_bmm GpuKit kit DTensor a DTensor b → ?DTensor

@ dtensor_conv2d GpuKit kit DTensor x DTensor w i ph i pw i sh i sw → ?DTensor

@ dtensor_conv2d_b GpuKit kit DTensor x DTensor w DTensor bias i ph i pw i sh i sw → ?DTensor

@ dtensor_maxpool2d GpuKit kit DTensor x i kh i kw i sh i sw i ph i pw → ?DTensor

Max pool over [C,H,W]: kernel kh×kw, strides sh/sw, symmetric pads.

@ dtensor_softmax GpuKit kit DTensor a → ?DTensor

@ dtensor_sum GpuKit kit DTensor a → ?f

Sum of every element (downloads one scalar; accumulates in the buffer's element type — true float32 for TE_F32).

---

ops.nu

tensor/ops.nu — broadcasting elementwise, unary maps, matmul and reductions.

Elementwise binary ops broadcast with numpy rules (align trailing dims, size-1 stretches). matmul runs on the GPU via gpukit for large f64 problems and on a plain triple loop otherwise — identical results either way. F32 tensors keep their outputs on the f32 grid.

API

: ~ i g_t_gpu 0 // 0 unprobed, 1 ready, -1 unavailable/disabled

── GPU singleton (matmul fast path) ───────────────────────────────── tensor_matmul routes >=100k-flop products through a gpukit kernel. Since the gpukit 0.4.2 fmad fix the kernel is BIT-IDENTICAL to the sequential host loop, so the route can never change a result — it is pure speed. The rules, stated rather than implied:

• AUTO only on a REAL CUDA device. The probe used to fall back to the

gpu package's CPU backend, which silently spawned a C++ compile + dlopen on the first big matmul of a GPU-less machine — a hidden side effect with no clear win over the plain loop. Gone: auto now requires gk_backend == cuda.

• NURL_TENSOR_GPU=0 disables the auto-probe outright (env opt-out).
• tensor_use_gpu(kit) opts IN explicitly with a caller-opened kit —

any backend, the caller decided. tensor holds one owner of it until tensor_gpu_close.

• tensor_gpu_off() disables the route (closing an open kit);

tensor_gpu_active() reports the current state.

: ~ i g_t_kit 0

The adopted kit: one owner of it, kept in a global for as long as the route is open (tensor_gpu_close lets it go).

@ tensor_use_gpu GpuKit kit → v

Opt IN with a caller-opened kit (any backend — the caller chose it). tensor holds the kit (one more owner of it) until tensor_gpu_close / tensor_gpu_off.

@ tensor_gpu_off → v

Disable the matmul GPU route for this process (idempotent).

@ tensor_gpu_active → b

Is the route currently open?

@ _t_bshape ( Vec i ) a ( Vec i ) b → ?( Vec i )

@ _t_eff_strides ( Vec i ) shape i nd → ( Vec i )

Effective strides of shape aligned into nd output dims (0 where broadcast).

@ tensor_add Tensor a Tensor b → ?Tensor

@ tensor_sub Tensor a Tensor b → ?Tensor

@ tensor_mul Tensor a Tensor b → ?Tensor

@ tensor_div Tensor a Tensor b → ?Tensor

@ tensor_maximum Tensor a Tensor b → ?Tensor

@ tensor_minimum Tensor a Tensor b → ?Tensor

@ tensor_pow Tensor a Tensor b → ?Tensor

@ tensor_adds Tensor t f s → Tensor

@ tensor_muls Tensor t f s → Tensor

@ tensor_subs Tensor t f s → Tensor

@ tensor_divs Tensor t f s → Tensor

@ tensor_neg Tensor t → Tensor

@ tensor_abs Tensor t → Tensor

@ tensor_exp Tensor t → Tensor

@ tensor_log Tensor t → Tensor

@ tensor_sqrt Tensor t → Tensor

@ tensor_relu Tensor t → Tensor

@ tensor_sigmoid Tensor t → Tensor

@ tensor_tanh Tensor t → Tensor

@ tensor_matmul Tensor a Tensor b → ?Tensor

@ tensor_transpose Tensor t → ?Tensor

@ tensor_permute Tensor t ( Vec i ) perm → ?Tensor

General axis permutation. perm is a length-ndim ordering of the axes.

@ tensor_sum Tensor t i axis b keepdim → Tensor

axis < 0 → reduce over everything (scalar result).

@ tensor_max Tensor t i axis b keepdim → Tensor

@ tensor_min Tensor t i axis b keepdim → Tensor

@ tensor_prod Tensor t i axis b keepdim → Tensor

@ tensor_mean Tensor t i axis b keepdim → Tensor

@ _t_batch_eff ( Vec i ) shape i ndb → ( Vec i )

Batch strides of shape (its first ndim-2 dims), aligned into ndb output batch dims with broadcasting (0 where the batch dim is 1 or padded).

@ _t_take_head ( Vec i ) shape i cnt → ( Vec i )

@ tensor_bmm Tensor a Tensor b → ?Tensor

Batched matmul: [..,M,K] · [..,K,N] → [..,M,N], batch dims broadcast.

@ tensor_softmax Tensor t i axis → Tensor

Numerically-stable softmax along one axis.

@ tensor_slice Tensor t ( Vec i ) starts ( Vec i ) stops → ?Tensor

Slice: out[d] spans [starts[d], stops[d]) of each dim.

@ tensor_concat2 Tensor a Tensor b i axis → ?Tensor

Concatenate two tensors along axis (shapes equal except along axis).

@ tensor_argmax Tensor t i axis b keepdim → Tensor

@ tensor_argmin Tensor t i axis b keepdim → Tensor