tooling(qa): roll-up view over qa-run records — selector, latest verdict, staleness

[claude]

Fixes#9486

Adds scripts/qa/qa-rollup.mjs — one command that answers "what is the latest verdict for every checklist selector, and is it stale?", replacing a 23-issue manual read.

It is a generated view, not a tracker: no cache, no checked-in snapshot, no last-run file. It reads the API, prints the matrix, and exits. Verified — after a full run git status shows nothing new. The qa-run issues stay the only source of truth.

node scripts/qa/qa-rollup.mjs # the matrix
node scripts/qa/qa-rollup.mjs --target REF # staleness against another ref
node scripts/qa/qa-rollup.mjs --json # same model, machine-readable
node scripts/qa/qa-rollup.mjs --self-test # 60 assertions, no network

Three assumptions were measured, not inherited — each one changed the design

1. The title convention is not followed by anything. Three conventions are in play, and the records follow none of the documented ones:

source	shape	records following it
.claude/skills/checklist-test/SKILL.md	5 fields, standalone judged/total, no counts	0 / 23
the card's description	6 fields, standalone judged/total, plus counts	0 / 23
what records actually write	5 fields, judged/total folded into a parenthetical on the selector	23 / 23

That parenthetical appears in five distinct phrasings (18/18, 77 items, FULL area, 6/14 items consulted, 5 of 10 items) and is absent entirely on one record. The parser accepts the shape the records have; anything it cannot parse is printed in a "Not parsed" section, never dropped. A roll-up that silently omits records is the "derived list read as authoritative" defect this repo has paid for repeatedly (#9294, #9331, #9503, #9590). Today that section reads "none" — all 23 parse.

2. The bodies do not have one shape, so they are deliberately not parsed. Measured across the corpus: 9 of 23 records carry no markdown table at all (the 2026-08-11 wave is prose with ## PASS — 4 items headings); the other 14 carry 26 distinct table header shapes. Worse than heterogeneous — most are clause-level, not item-level (clause / verdict / oracle evidence appears 38 times). They use the word "verdict" for a different unit, so a parser that greps verdict cells counts clauses and reports them as items: a confidently wrong matrix, which is worse than no matrix.

So the script reads the field that is actually a contract — the title's own counts, authored by the person who ran the tests — and the output states, every run, that it did not read the bodies. Nobody can mistake this view for a body-derived one.

3. Staleness is three-valued, because a shallow clone cannot always tell.git merge-base --is-ancestor needs the sha present with history behind it. This container clones shallow (graft boundary 713ccbc95, 2026-08-16), and the 92f26f75 wave predates it — so --is-ancestor answers false for commits that almost certainly are ancestors. Reading that as "not behind, therefore fresh" would have rendered the nine stalest records in the corpus as fresh, silently. Freshness is therefore current / stale / unknown, with unknown carrying its reason (shallow / unreachable / diverged) and rendering as ?. "Could not tell" is never counted as fresh.

Rate limit (the card's untested assumption). Measured: REST core, 1 call for the whole 23-record corpus, against a 15000/hr quota (14967 remaining at the time). Deliberately not the search API (30/min) and not GraphQL, whose quota was measurably under pressure (2699/5000 remaining) while this was written. Comments are never fetched, because bodies are not parsed.

What the view surfaces today

Real signals visible in the first run, none of which any current view shows:

• 5 areas (ai, attachments-storage, cli, i18n, search) have no verdict newer than 2026-08-11, and honestly render ? shallow rather than a fabricated freshness.
• dashboards (10 items) has never been run — an area in the checklist that no record names. Reported as absence of a verdict, not as a verdict of NOT-RUN.
• records-forms is titled "(FULL area)" but judged 13 of 33 items, rendered 13/33*. The * marks a denominator taken from the checklist rather than claimed by the record — the two provenances are kept visually distinct precisely so this gap is legible instead of reassuring.
• An undeclared verdict bucket renders —, never 0. A record that did not declare NOT-RUN has not declared zero of them.

Notes for review

• No package.json entry, on purpose. The card asks for a view, not a gate; wiring one would expand the verification surface and touch a file two open PRs hold. I checked the conflict factually, as dispatched: ci: at most one open PR may modify a declared single-writer path #9598 adds one line to the same scripts block, chore(deps)(deps-dev): bump the development-dependencies group across 1 directory with 11 updates #9466 bumps devDependencies there. ci: at most one open PR may modify a declared single-writer path #9598 introduces a single-writer-path guard, but its declared list contains only .objectui-sha — root package.json is not declared, so that gate would not have blocked this. The decision is scope discipline, not the guard. Wiring --self-test as a check:* family once ci: at most one open PR may modify a declared single-writer path #9598/chore(deps)(deps-dev): bump the development-dependencies group across 1 directory with 11 updates #9466 land is a clean follow-up.
• A governed surface is stale and I did not touch it. The title convention documented at .claude/skills/checklist-test/SKILL.md:126 no longer matches what any record writes. That file is human-merge-only, so this is reported, not edited.
• skip-changeset: this PR adds one repo-internal tooling script and publishes nothing.

Gates

Run after the final commit, all green at 506c30962:

node scripts/qa/qa-rollup.mjs --self-test ✓ 60 assertions over 7 real-corpus title shapes
node scripts/check-nul-bytes.mjs ✓ 6172 files, no raw ASCII control bytes
npx eslint scripts/qa/qa-rollup.mjs ✓ exit 0
node scripts/pm/dispatch-gates.mjs scripts/qa/qa-rollup.mjs
→ no path-derived gate family

Reverse-verified against real data: re-running with --target e4e5c6e3 flips the 14 rows measured on that sha from STALE (122 behind) to current, while the 5 shallow-unknown rows correctly stay ? — confirming the freshness column reads live git ancestry rather than a fixed label.

---

Generated by Claude Code