NURLNURL registrynurl-lang.org →

← nurl-cov

nurl-cov 0.2.0 API

main.nu

nurl-cov — test-coverage mapper for NURL.

nurl-cov run build tests/ instrumented, run, report nurl-cov report <dir> report on coverage graphs already there nurl-cov gcov <dir|file.gcno> gcov-style annotated source

Options that shape the output apply to run and report alike: --uncovered list the line ranges nothing executed --lcov FILE write an LCOV tracefile --html FILE write one self-contained HTML report --json FILE write machine-readable JSON --fail-under PCT exit non-zero below this line coverage --include PREFIX keep only files under PREFIX (repeatable) --all keep every file, the stdlib included

nurl-cov run --fail-under 80 --lcov coverage.info

Exit codes: 0 all good · 1 below the floor, or a test failed · 2 usage or a broken build. A gate that cannot tell "untested" from "failed to build" is not a gate, so those are different codes and different words.

API

: s NURLCOV_VERSION 0.2.0``

@ main → i

model.nu

nurl-cov/model.nu — one coverage picture from many test binaries.

A suite is not one program. Each test is built and run on its own, so each leaves its own pair of coverage graphs, and each sees only the part of the package it exercises. The number a person wants is the union: a line covered by the parser test and the writer test is covered, and one covered by neither is the finding.

Merging is addition, on three axes:

traffic of every test,

notes fix, which every binary built from the same source agrees on,

A file no test ever compiled in is simply absent, and absent is not the same as uncovered: this model reports what the tests built. runner.nu is what makes sure they build everything the package ships.

API

: CovFn

: CovFn {
    String name
    i line
    i called  // entries, summed over every object
}

: i CBR_W 3

Branch table stride.

: i CBR_LINE 0

: i CBR_IDX 1 // ordinal within the line, as the notes numbered it

: i CBR_COUNT 2

: CovFile

: CovFile {
    String path
    ( Vec i ) exists  // indexed by line number: 1 when the line has code
    ( Vec i ) count  // indexed by line number: executions
    ( Vec i ) branches  // stride CBR_W
    ( Vec CovFn ) funcs
}

: CovImpl

: CovImpl {
    ( Vec CovFile ) files
    i objects  // how many coverage objects were folded in
}

: Cov { s ctl }

A Cov is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same state, and the last owner releases it.

@ Cov_share Cov h → Cov

@ Cov_drop sink Cov h → v

@ cov_objects Cov c__h → i

How many coverage objects were folded in.

: CovStat

: CovStat {
    i lines_found
    i lines_hit
    i branches_found
    i branches_hit
    i funcs_found
    i funcs_hit
}

@ cov_new → Cov

@ cov_free sink Cov c → v

The files (and everything they hold) are dropped with their Vec. Let go of c now rather than at the end of its owner's scope.

@ cov_file_count Cov c__h → i

@ cov_file_path Cov c__h i idx → s

@ cov_file_idx Cov c__h s path → i

The row for path, created when this is the first object to mention it.

@ cov_add_object Cov c__h GcovObj o → v

@ cov_keep_only Cov c__h ( Vec String ) prefixes → v

@ cov_sort Cov c__h → v

Order the report the way a person reads it: by path.

@ cov_file_stat Cov c__h i idx → CovStat

@ cov_total Cov c__h → CovStat

@ cov_max_line Cov c__h i idx → i

@ cov_line_exists Cov c__h i idx i line → b

@ cov_line_count Cov c__h i idx i line → i

@ cov_branch_rows Cov c__h i idx → i

@ cov_branch_field Cov c__h i idx i row i field → i

@ cov_fn_rows Cov c__h i idx → i

@ cov_fn_name Cov c__h i idx i row → s

@ cov_fn_line Cov c__h i idx i row → i

@ cov_fn_called Cov c__h i idx i row → i

@ cov_pct_tenths i hit i found → i

Percentages in tenths, so a report prints one decimal without floating point. An empty denominator is 100%: there is nothing left uncovered.

lines.nu

nurl-cov/lines.nu — from basic blocks to "this line ran N times".

A line is not a block, and the answer is not "add up the blocks that touch it". A condition and the two arms it guards can all carry the same source line; adding them reports the line running twice as often as the function was entered. The rule that IS correct — and the one gcov itself implements — measures flow instead of blocks:

the count of a line is the traffic ENTERING that line's blocks from outside them, plus the traffic that goes round in circles INSIDE them.

The first part is a sum over incoming arcs whose source is not itself on the line (the entry block has no predecessors, so its outgoing arcs stand in). The second part is what makes a one-line loop report the number of iterations rather than the single entry into it; finding it is cycle cancelling over the line's own sub-graph, repeatedly draining the cheapest arc of some cycle until no cycle is left.

Both halves are needed. Without the first, a loop header reports its back-edge only; without the second, it reports the one entry.

The same table drives the annotated listing and the merged report, so there is exactly one implementation of the rule in this package.

API

: i LOC_W 3

Occurrence table stride: one row each time a block names a line.

: i LOC_LINE 0

: i LOC_FN 1

: i LOC_BLK 2

: i LBR_W 3

Branch table stride.

: i LBR_LINE 0

: i LBR_COUNT 1 // this arc's traffic

: i LBR_TOTAL 2 // traffic over all arcs leaving the same block

: i LFN_W 2

Function-start table stride.

: i LFN_LINE 0

: i LFN_FN 1

: LineTabImpl

: LineTabImpl {
    i src  // which of the object's source files this describes
    i maxline
    ( Vec i ) exists  // per line: 1 when some block names it
    ( Vec i ) count  // per line: executions
    ( Vec i ) br  // stride LBR_W, in gcov's print order
    ( Vec i ) fnrow  // stride LFN_W: functions starting on a line
    ( Vec i ) occ  // stride LOC_W
    ( Vec i ) occ_next  // next occurrence on the same line, + 1
    ( Vec i ) head  // per line: first occurrence + 1
    ( Vec i ) tail  // per line: last occurrence + 1
}

: LineTab { s ctl }

A LineTab is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same state, and the last owner releases it.

@ LineTab_share LineTab h → LineTab

@ LineTab_drop sink LineTab h → v

@ linetab_maxline LineTab t__h → i

The table's columns, lent: they live as long as the LineTab does.

@ linetab_exists LineTab t__h → ( Vec i )

@ linetab_count LineTab t__h → ( Vec i )

@ linetab_br LineTab t__h → ( Vec i )

@ linetab_fnrow LineTab t__h → ( Vec i )

@ linetab_free sink LineTab t → v

Let go of t now rather than at the end of its owner's scope.

@ lines_build GcovObj o i src → LineTab

jsonout.nu

nurl-cov/jsonout.nu — the machine-readable shape of the same numbers.

For a CI step that wants to compare two runs, post a comment, or gate on something more specific than one percentage. The line map is an object keyed by line number, so a consumer can ask about a line directly instead of scanning an array.

{ "objects": 7, "totals": { "lines_found": …, "lines_hit": …, … }, "files": [ { "path": "src/x.nu", "lines_found": …, "lines_hit": …, "lines": { "12": 4, "13": 0 }, "branches": [ { "line": 12, "index": 0, "count": 4 } ], "functions": [ { "name": "f", "line": 3, "called": 4 } ] } ] }

API

@ json_render Cov c → String

lcov.nu

nurl-cov/lcov.nu — the tracefile everything else already reads.

LCOV's .info format is the lingua franca of coverage: genhtml renders it, Codecov and Coveralls ingest it, and editors draw the gutter from it. Emitting it is what lets a NURL package sit in a CI pipeline that was never taught anything about NURL.

SF:<path> one section per source file FN:<line>,<name> a function and where it starts FNDA:<count>,<name> how often it was called FNF/FNH functions found / hit BRDA:<line>,0,<n>,<count> a branch outcome ("-" when never reached) BRF/BRH branches found / hit DA:<line>,<count> a line and its count LF/LH lines found / hit end_of_record

The block field of BRDA is always 0: NURL has no notion a consumer could use it for, and lcov itself only requires the triple to be stable.

API

@ lcov_render Cov c → String

report.nu

nurl-cov/report.nu — the table a person reads, and the lines they act on.

The summary answers "how much of this package do the tests touch"; the uncovered listing answers the question that actually changes what someone does next: WHICH lines. A percentage alone has never told anybody where to write a test, so --uncovered prints the line ranges, collapsed, in the order they appear in the file.

Colour is used only to rank: red under the floor, yellow near it, green above. It is switched off when stdout is not a terminal, and by $NO_COLOR, so redirecting the report into a file gives plain text.

API

: i REP_NAME_W 44

: RepStyle

: RepStyle {
    b colour
    i warn_tenths  // below this, a file is flagged
    i floor_tenths  // below this, a file is a failure
}

@ report_style b colour i floor_tenths → RepStyle

@ report_colour_wanted → b

Colour is a courtesy to a human at a terminal, never something a pipe has to strip back out.

@ report_display s path s root → s

Coverage graphs name their sources absolutely, because the compiler wrote them that way. A report read next to the code wants the path the reader would type, so the working directory is taken off the front.

@ report_render Cov c RepStyle st s root → String

@ report_uncovered Cov c s root → String

The lines nothing ran, collapsed into ranges. This is the part of a coverage report that is actionable: it names the work.

html.nu

nurl-cov/html.nu — one self-contained page you can send to someone.

A coverage report is read by opening it, scrolling to the red, and writing a test. That wants the source next to the counts, not a percentage in a table — so this emits the summary AND every file's text, line by line, with its count in the gutter and uncovered lines marked.

One file, no assets, no JavaScript beyond a details/summary toggle the browser implements itself. It opens from a file:// URL, survives being mailed around, and can be published as a CI artifact without a server.

API

@ html_render Cov c i floor_tenths s title s root → String

runner.nu

nurl-cov/runner.nu — build the suite instrumented, run it, collect the graphs.

A NURL package's tests are programs: one .nu per file under tests/, each importing the package's own modules and exiting non-zero on failure. Measuring them means building each one again with the compiler's coverage instrumentation, running it, and reading what it leaves behind.

Two flags make the difference between a report and a flattering one:

-O0 the optimiser is free to fold, duplicate and reorder blocks, and a line's count stops being a line's count. --no-dce dead-code elimination removes functions nothing calls — which is EXACTLY the code a coverage report exists to find. Without it the report cannot see what it is for: on a small sample here that one flag moved the number from 88.9% to an honest 72.7%.

The build driver is resolved the way nurlpkg test resolves it, so a checkout and an installed toolchain both work without configuration.

API

: RunOne

: RunOne {
    String name
    b built
    b ran
    i code
    String detail  // compiler or runtime output when something went wrong
}

: RunResult

: RunResult {
    ( Vec RunOne ) tests
    i built
    i failed  // tests that exited non-zero
    i broken  // tests that would not build or would not run
}

@ runresult_free sink RunResult r → v

A plain value: the tests (names and details) are dropped with their Vec. Let go of r now rather than at the end of its owner's scope.

@ runner_driver → String

The build driver, in the order nurlpkg test looks for one: an explicit override, then a checkout you are standing in, then the installed tool.

@ runner_find_tests s dir → ( Vec String )

Every .nu directly under dir, in name order so a run is repeatable.

@ runner_one String driver String testpath String stem s workdir → RunOne

Build and run one test. The binary and its coverage graphs go to workdir; the compiler writes <stem>.gcno there, and the program writes <stem>.gcda beside it on exit — both paths are absolute in the generated module, so it does not matter where the program is started.

@ runner_run_all String driver ( Vec String ) tests s testdir s workdir → RunResult

@ runner_clean s dir → v

Start from a clean slate.

A .gcda ACCUMULATES: running an instrumented program again adds to the counters already there. That is the right behaviour for a program you are exercising by hand, and the wrong one for a suite you are measuring — the second nurl-cov run would report the first run's traffic too, and a .gcno left over from a test that has since been deleted would keep contributing coverage for code nothing tests any more. Both are silent, and both flatter.

@ runner_objects s dir → ( Vec String )

Every coverage object in dir: a .gcno the compiler wrote, paired with the .gcda its program left — or not, when the program never ran.

@ runner_data_path s notes → String

gcovtext.nu

nurl-cov/gcovtext.nu — the annotated source listing, in gcov's own format.

One listing per source file: every line of the original, prefixed by how many times it ran. A line with no code gets -, a line with code that never ran gets #####, and that column is the whole point — it is where you look to find what the tests never touched.

function classify called 2 returned 100% blocks executed 70% 2: 1:@ classify i n → s { 2: 2: ? < n 0 { ^ negative } {} branch 0 taken 2 branch 1 taken 0 #####: 5: ^ large

The layout is deliberately byte-compatible with llvm-cov gcov -b -c. That is not nostalgia. It means this reader can be checked against an independent implementation of the same files, which is the only way to know that numbers nobody can compute by hand are right — and the package's test suite does exactly that, over the compiler's own corpus.

This renders ONE object. Merged, multi-binary reporting is report.nu's job; it cannot answer per-block questions like "blocks executed" and does not pretend to.

API

@ gcovtext_render GcovObj o i src → String

Render one source file's annotated listing. A file whose text cannot be read still gets a listing: the counts are the part that matters, and a missing source is not a reason to withhold them.

@ gcovtext_pct i part i whole → i

gcov's percentage: truncating, except that anything above zero but below one percent reports as 1 rather than rounding away to nothing.

gcov.nu

nurl-cov/gcov.nu — read the compiler's coverage graphs, in pure NURL.

nurlc --coverage=PREFIX asks LLVM's GCOV pass to write two files:

PREFIX.gcno the NOTES: one control-flow graph per function, and the source lines each basic block came from. Written once, at compile time. PREFIX.gcda the DATA: one counter per instrumented arc, written when the instrumented program exits. Re-running the program accumulates into the same file.

Neither file holds a per-line count. The notes hold a spanning tree over each function's arcs, and only the arcs OUTSIDE that tree are counted — that is the whole point of the encoding, and it is why a coverage reader has real work to do. This module does that work:

  1. parse both files (a tagged record stream, 32-bit words, LE),
  2. hand each counted arc its counter,
  3. solve the flow graph for every remaining arc and every block count,
  4. hand back the per-block counts with the lines each block covers.

Turning that into per-line numbers is model.nu's job; rendering it the way gcov does is gcovtext.nu's.

Format: the version word is "408*" — GCC 4.8's layout, which is what LLVM's GCOV pass emits and what this reader implements. A file that announces a different version is rejected rather than guessed at.

Memory model: gcov_read returns a GcovObj handle — every copy is the same object and the last owner releases it (gcov_free is an optional early release). Every table inside it is flat — parallel ( Vec i ) columns indexed by function/block/arc number — so nothing owns anything twice.

API

: i GCOV_TAG_FUNCTION 0x01000000

Record tags, as they appear in both files.

: i GCOV_TAG_BLOCKS 0x01410000

: i GCOV_TAG_ARCS 0x01430000

: i GCOV_TAG_LINES 0x01450000

: i GCOV_TAG_COUNTER 0x01a10000

: i GCOV_TAG_PROGRAM 0xa3000000

: i GCOV_ARC_ON_TREE 1

Arc flags.

: i GCOV_ARC_FAKE 2

: i GFN_W 13

Function table stride. One row per function.

: i GFN_IDENT 0

: i GFN_LCS 11 // line-number checksum, as the notes recorded it

: i GFN_CCS 12 // control-flow checksum

: i GFN_SRC 1 // index into files``

: i GFN_LINE 2

: i GFN_NBLOCK 3

: i GFN_BLK_OFF 4 // first row in blk_count``

: i GFN_ARC_OFF 5 // first row in arcs (already multiplied by GARC_W)

: i GFN_ARC_N 6

: i GFN_BL_OFF 7 // first row in blines (already multiplied by GBL_W)

: i GFN_BL_N 8

: i GFN_CTR_OFF 9 // first slot in counters``

: i GFN_CTR_N 10

: i GARC_W 5

Arc table stride.

: i GARC_SRC 0

: i GARC_DST 1

: i GARC_FLAGS 2

: i GARC_COUNT 3

: i GARC_VALID 4

: i GBL_W 3

Block-line table stride: which source line a block covers.

: i GBL_BLOCK 0

: i GBL_SRC 1

: i GBL_LINE 2

: i GCOV_MAX_LINE 16777216

A line number no source file reaches. The per-line tables are indexed BY line number, so an untrusted number decides how much memory this reader asks for: one flipped byte in a notes file named line 1970155382, and a dense table that long is tens of gigabytes and a hang. The largest file in this project is 33795 lines. Sixteen million is a ceiling no real source reaches and every corrupt one blows straight through.

: GcovObjImpl

: GcovObjImpl {
    String notes  // path of the .gcno that was read
    String data  // path of the .gcda, or empty when none was found
    i stamp
    i runs  // how many times the instrumented program has exited
    b has_data  // false → the program never ran; every counter is 0
    ( Vec String ) files  // source files this object mentions
    ( Vec i ) fns  // stride GFN_W
    ( Vec String ) fn_names
    ( Vec i ) blk_count  // solved execution count, per block, per function
    ( Vec i ) arcs  // stride GARC_W
    ( Vec i ) blines  // stride GBL_W
    ( Vec i ) counters  // raw arc counters from the .gcda
    // Edge index: for every block, the arcs entering and leaving it, in
    // the order the notes stored them. Without it, finding a block's
    // arcs means scanning its function's whole arc table, and a large
    // module turns into an O(n^2) walk.
    ( Vec i ) pred_head  // per block slot: first arc slot + 1
    ( Vec i ) pred_next  // per arc slot: next arc slot + 1
    ( Vec i ) pred_tail
    ( Vec i ) succ_head
    ( Vec i ) succ_next
    ( Vec i ) succ_tail
}

: GcovObj { s ctl }

A GcovObj is a handle on its state in an rcbox (stdlib/core/rcbox.nu): every copy is the same state, and the last owner releases it.

@ GcovObj_share GcovObj h → GcovObj

@ GcovObj_drop sink GcovObj h → v

: | GcovErr

: | GcovErr {
    GcovNoNotes  // the .gcno could not be read
    GcovBadMagic  // not a coverage file at all
    GcovBadVersion  // a GCOV layout this reader does not implement
    GcovTruncated  // a record runs past the end of the file
    GcovStampMismatch  // .gcda belongs to a different build than the .gcno
    GcovChecksumMismatch  // a function's own checksums disagree across the pair
    GcovBadLine  // a line number no source file could have
    GcovBadBlock  // a record names a block the function does not have
}

@ gcov_err_name GcovErr e → s

@ gcov_fn_count GcovObj o__h → i

@ gcov_fn_name GcovObj o__h i fi → s

@ gcov_fn_src GcovObj o__h i fi → i

@ gcov_fn_line GcovObj o__h i fi → i

@ gcov_fn_nblocks GcovObj o__h i fi → i

@ gcov_fn_blk_off GcovObj o__h i fi → i

Where this function's blocks start in the object-wide block table. An index over all blocks needs one flat numbering, and this is it.

@ gcov_total_blocks GcovObj o__h → i

@ gcov_total_arcs GcovObj o__h → i

@ gcov_block_count GcovObj o__h i fi i blk → i

@ gcov_file_count GcovObj o__h → i

@ gcov_runs GcovObj o__h → i

@ gcov_notes_path GcovObj o__h → s

@ gcov_data_path GcovObj o__h → s

@ gcov_file_path GcovObj o__h i idx → s

@ gcov_fn_bl_first GcovObj o__h i fi → i

The block-line rows of one function, as a half-open range over blines.

@ gcov_fn_bl_end GcovObj o__h i fi → i

@ gcov_bl_block GcovObj o__h i row → i

@ gcov_bl_src GcovObj o__h i row → i

@ gcov_bl_line GcovObj o__h i row → i

@ gcov_fn_arc_first GcovObj o__h i fi → i

@ gcov_fn_arc_end GcovObj o__h i fi → i

@ gcov_arc_src GcovObj o__h i ai → i

@ gcov_arc_dst GcovObj o__h i ai → i

@ gcov_arc_flags GcovObj o__h i ai → i

@ gcov_arc_count GcovObj o__h i ai → i

@ gcov_arc_is_branch GcovObj o__h i ai → b

An arc the instrumenter treated as a real branch. Fake arcs (the ones LLVM adds for abnormal exits) are not branches a test can take.

@ gcov_edge_first GcovObj o__h i fi i blk i side → i

The first arc entering (side 0) or leaving (side 1) a block, as a chain cursor: non-zero is an arc slot plus one, zero is the end.

@ gcov_edge_next GcovObj o__h i cursor i side → i

@ gcov_edge_arc i cursor → i

The arc a cursor points at, as an index into the arc table.

: i GST_W 6

Stack frame layout for the walk below. An explicit stack, not recursion: a deeply nested function would otherwise decide how much stack a coverage report needs.

The frame carries a CURSOR into the block's arc chain rather than a count of how many arcs it has consumed. Counting means re-walking the chain to find the next one, which is quadratic in a block's degree — invisible on ordinary code, and a hang on a graph where one block has thousands of arcs pointing at it.

: i GST_BLK 0

: i GST_PRED 1 // arc index we arrived by, -1 at a root

: i GST_INDST 2 // did we arrive along that arc's direction?

: i GST_SIDE 3 // -1 not started, 0 walking the arcs in, 1 the arcs out

: i GST_CUR 4 // chain cursor, 0 when this side is exhausted

: i GST_EXCESS 5

@ gcov_solve GcovObj o__h → v

Without data there is nothing to solve: every counter is zero, every block is zero, and the report says so.

@ gcov_new → GcovObj

@ gcov_free sink GcovObj o → v

Let go of o now rather than at the end of its owner's scope.

@ gcov_read s notes_path s data_path → !GcovObj GcovErr

Read one coverage object: the notes, and the data if it is there.

A missing .gcda is NOT an error — it is the answer "this program was built but never run", and a coverage report has to be able to say that. Every counter then reads zero, which is exactly what it means.