fix(ci): make the test-completeness guard see a package that reported nothing

[os-zhuang]

Part of #10032

check-test-completeness.mjs builds its rows by regex over vitest summary lines present in the log. A package that emits no summary at all contributes no row, so it is neither counted nor missed — it is invisible. The guard's green therefore means "every package that reported was internally consistent", never "every package on the shard reported", and ci.yml's own note read that green as "so these are real test failures". That inference sent one triage to a wrong hypothesis and kept it there.

This feeds the guard the shard's scheduled package set, so a scheduled-but-silent package becomes a named red.

What changed

• scripts/check-test-completeness.mjs takes --scheduled (ci.yml's $RUNNER_TEMP/shard-packages.txt) and --package-list (the turbo ls document it was sharded from, which resolves each name to a directory). Both or neither; one without the other is a usage error.
• .github/workflows/ci.yml passes both in the Test Core guard step, and the step's comment — the load-bearing sentence that documented the false dichotomy — is rewritten to match the guard's new meaning.

⛔ scripts/partition-test-shards.mjs is not touched. It is the fenced surface of #10149. This change only reads the file it produces.

⚠️ Patch round 1 — the first version produced a false positive in CI, and why

Test Core (1/3) on 487f6228 went red on this PR: the guard said @objectstack/spec "reported nothing" while the same log carried Test Files 415 passed (415) and Tests 11045 passed (11045).

Root cause, measured on turbo 2.10.10 in both orders. The first version read package attribution only from the <pkg>:test: line prefix. turbo uses stream log order locally, which prefixes every line — but switches to grouped order under GitHub Actions, which emits a group header and no per-line prefix at all:

::group::@objectstack/spec:test <- GitHub renders this as ##[group]
Test Files 415 passed (415)
Tests 11045 passed (11045)
::endgroup::

Grouped is therefore the only shape CI ever writes. The prefix-only join attributed nothing there, so it would have reddened every multi-package shard, not merely the single-task one that happened to be affected — reproduced locally with three packages in grouped order, where all three summaries arrive anonymous.

⚠️ One correction to the triage note on the issue: @objectstack/spec does not appear only once in that job log. ##[group]@objectstack/spec:test is at line 4305 of the ~5000-line log; the "appears exactly once" reading came from a 400-line tail window that starts below the header. That header is precisely what makes the fix possible.

The fix. The group header is turbo's own statement of whose output follows, so it is now the primary attribution, with an explicit line prefix still winning when present. Only a :test group attributes — :build groups and GitHub's own Run <script> step groups lend nothing.

And it does not guess. A summary neither spelling can attribute is a backstop for a third log shape: one remaining candidate is unambiguous and is accepted; more than one is not graded at all and says so out loud. Attributing a stray summary to the wrong package would mark a silent package as having reported — worse than the gap.

Fixture coverage is extended, since every original leg used stream-order logs. The self-test now pins both orders, including the exact three lines from the failing CI job, the grouped multi-package shape, and the group-leak / build-group / step-group cases.

Two measured false-red sources shape the rules

A naive scheduled-minus-reported join is a false-red machine. Both measured before the rules were written:

• Nothing to run.shard-packages.txt lists every package on the shard, including ones turbo runs no test task for. 5 of this repo's 77 packages declare no test script; 16 more run vitest run --passWithNoTests, which prints no summary when a package owns no test files. So a package is expected to report only if it has both a test script and at least one test file.
• Never reached. turbo stops scheduling on the first failure. Measured: one failing task in a 4-task run printed Tasks: 1 successful, 4 total, and the two cancelled packages produced no summary. Charging those to the guard would put noise on every ordinary red suite.

So a missing summary is red only under one of two rules — turbo named the package in its Failed: roster, or the run completed with every task successful and a package still reported nothing. A package the run never reached is a note, never red.

Verification — five legs, both log shapes, plus the real CI log

⛔ A green guard proves nothing here, since the subject is a guard that was green while blind. So the invisible case is constructed in both log orders, and origin/main is run against the same logs as a control.

leg	log	expected	result
A	grouped (CI shape), two silent failures	RED, naming both	exit 1, names sdui-parser + embedder-openai
B	grouped (CI shape), complete pass	GREEN	exit 0, 3 of 3 scheduled package(s) reported
C	stream (local shape), two silent failures	RED, naming both	exit 1, names both
D	stream, complete pass	GREEN	exit 0, 3 of 3 … reported
E	the real CI log that broke it	GREEN	exit 0, 1 of 1 … reported; 11045 test(s) declared and all accounted for

Before/after on the real bytes. The failing job's test-step output was extracted from the job log (timestamps stripped, i.e. what test-core.log holds) and replayed against both versions from inside the repo:

487f6228 (shipped): check-test-completeness: 1 package was scheduled on this shard and reported no vitest summary
• @objectstack/spec -- the run completed with every task successful, and this package reported nothing [exit 1]
this commit: check-test-completeness: OK (1 of 1 scheduled package(s) reported, 0 had nothing to run,
0 never reached; 11045 test(s) declared and all accounted for). [exit 0]

The true positive survives.origin/main's guard on the same two failing logs is green in both orders (OK (1 package(s), 356 test(s) …)) — still blind — while this version is red. The value is intact.

Anti-noise. On an aborted run with two no-script packages scheduled, only the genuinely silent failed package is red; cancelled ones are notes and no-script ones are silently exempt.

Backward compatibility. Without the new flags, stdout+stderr and exit code are byte-identical to origin/main on all five fixtures, so the Dogfood job's invocation is unaffected.

Cache replay, verified

Measured on turbo 2.10.10: a cached leg printed cache hit, replaying logs 07765db4f3648082 followed by both summary lines and >>> FULL TURBO. A >>> FULL TURBO shard therefore still reports and does not go red. turbo.json sets no outputLogs suppression on test — the premise #10032 stated, re-verified.

The self-test runs on every invocation, not from a lint step

The rules are pure functions over strings and now carry ~45 assertions, running at the guard's own startup (~1ms) rather than from a lint.yml step: lint.yml is outside this change's declared file surface, and this repo has already paid for the alternative — partition-test-shards.mjs carried a --self-test that nothing ran from the day it was written, which lint.yml itself records as "a pin nobody runs is not a weaker pin, it is no pin". --self-test also stands alone for local use.

Scope

Option 1 of #10032 only. Option 2 is filed as #10203 with the measurements that changed its value — notably that it would have captured nothing in the incident that motivated it. Option 3 is out of scope per the card ("worth doing only if it recurs").

⛔ No root cause is claimed for the original zero-output event. It was never reproduced and is not diagnosed here; this only makes CI able to say that it happened, and to which package.

Gates

main merged first (04096f17). node scripts/pm/dispatch-gates.mjs with no path args, derived from the real diff at 7f71e56e, named the same 9 families. All green at that commit, each quoting its own verdict line:

gate	verdict line
check:cross-package-test-inputs	OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.
check:node-version	check-node-version: OK (29 setup-node step(s) across 26 workflow(s), all on Node 22).
check:required-contexts	green; still resolves ci.yml:test-gate → 'Test Core'
check:shard-attestation	check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s).
check:workflow-status-functions	OK (scanned 26 workflow file(s), 45 job(s), 23 job-level if: expression(s); 10 read needs.*.outputs.*, all naming a status function).
check-test-completeness.mjs (the guard itself)	check-test-completeness: self-test OK

Plus check:nul-bytes → OK (scanned 6075 text file(s) … no raw ASCII control bytes).

⚠️ Declared narrowing: @objectstack/spec's own suite (8m16s) was not run locally. It is pulled in only by spec's scripts/** turbo input glob, and no spec test reads this script — the sole repo reference is a prose comment in strict-object.test.ts. CI runs it regardless.

No changeset: the diff is scripts/ plus a workflow, and publishes nothing. skip-changeset applied.

---

Generated by Claude Code

---

Generated by Claude Code