Uh oh!
There was an error while loading. Please reload this page.
test(cli): derive the vitest tier population instead of freezing it in a list - #14717
Conversation
…n a list `INTEGRATION_FILES` was a hand-maintained copy of a fact already on disk, so it went stale whenever ANOTHER PR landed a qualifying test file. The pin that held list == predicate then fired inside the merge queue, against a `main` newer than any queued PR's own run, ejecting bystanders behind the root entry. `packages/cli/vitest-tiers.ts` now carries the predicate and derives the population at config load; the config hands the result to both projects. The predicate is unchanged (SPAWN or KERNEL) and re-deriving it over this tree reproduces the last hand-maintained list exactly: 72 of 230 files, none moved. The pin keeps the coverage cases, gains an end-to-end check that the derivation reaches vitest, and replaces the independence the frozen list gave with fixture sources for every signal and every tuned false positive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza
The section still described `INTEGRATION_FILES` as an explicit list the pin re-derives and compares. It now points at `vitest-tiers.ts` as the single statement of the predicate, records the merge-queue shape the derivation removes, and reports the measured population identity across the change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza
…test-tiers-partition-queue-eject
📓 Docs Drift CheckThis PR changes 1 package(s): 2 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
What this run could not see
Coarse fallback — 22 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 0a5cd0208ebceadd15f81d2c7e6ff8b806f6a28b && git checkout 0a5cd0208ebceadd15f81d2c7e6ff8b806f6a28b
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 75adf11da2a5ca5bae522dfc4209bd3bdd20f4c5 d29fbf2a78c6001fb1c864b254e878bf1e79382f && git checkout -B drift-repro 75adf11da2a5ca5bae522dfc4209bd3bdd20f4c5 && git merge --no-ff d29fbf2a78c6001fb1c864b254e878bf1e79382f
node scripts/docs-audit/affected-docs.mjs --json 75adf11da2a5ca5bae522dfc4209bd3bdd20f4c5
|
Uh oh!
There was an error while loading. Please reload this page.
Fixes#14554
The anchor named a test file. The thing behind the name is a maintenance
duplicate:
INTEGRATION_FILESinpackages/cli/vitest.config.tswas ahand-written copy of a fact already on disk, and the pin held the copy equal to
a predicate re-derived from the tree. That equality is a real invariant — but a
copy of the tree goes stale when someone else lands a qualifying test file,
and the pin that notices runs inside the merge queue, against a
mainthatis by construction newer than any queued PR's own run. GitHub stacks queue
entries, so one deterministic red ejects everything behind it.
Measured on the anchor: five ejections in a rolling 24 hours, one independent
hit, four bystanders touching no
packages/clipath at all.The premise, re-verified rather than taken on trust
mainand both files named in theseat's account are in the list today.
origin/mainreproduces that list exactly:72 of 230, zero missing, zero stale. So the list is in sync now — which is
precisely why closing the anchor on that basis would have been true of one
ejection and false about the shape.
What changed
packages/cli/vitest-tiers.tsintegrationTestFiles()which derives the population from itpackages/cli/vitest-tiers.fixtures.tspackages/cli/vitest.config.tsintegrationTestFiles(__dirname); header rewrittenpackages/cli/test/vitest-tiers-partition.test.tsA qualifying file arriving on
mainis now classified, not reported.⛔ Deliberately unchanged: the predicate is
SPAWN or KERNEL, character forcharacter. No tier means anything new, nothing is skipped, quarantined or
softened, and the list is not deleted in favour of one project — the ~24-minute
serialised run that justifies the split (#13504) is untouched.
Population — every unit accounted for
vitest list --filesOnly --project NAME, run on the pre-change base and on thisbranch:
The two file lists are byte-identical per tier (
diffreports 0 lines).No test file changes tier in this PR.
Read that as the point, not as a footnote. Nothing is re-partitioned here.
The tiers hold exactly the files they held, in exactly the tiers they held them,
and
pnpm testruns exactly what it ran. This is a change to how membership isMAINTAINED — a robustness change to the instrument — and the identical
populations are the evidence for that claim.
The mechanism, demonstrated in both directions
Both legs use one synthetic stand-in for "a qualifying file that landed on
main" — a test file constructing a kernel query, the sameobjectQLCtorsignal that produced the real ejection. It is added to the pre-change base tree
and to this branch, byte-for-byte the same file.
Direction 1 — a file merely new on
mainmust no longer redden.files that spawn the CLI or boot a kernel/driver but are NOT in INTEGRATION_FILES (add them), namingtest/queue-arrival-probe.e2e.test.ts [objectQLCtor]integrationtier (1 hit)The base failure is the incident reproduced: same assertion, same signal, on a
PR whose own diff is unrelated to the file.
Direction 2 — a file matching NO project must still redden.
The integration project's
includewas mutated to drop one entry while the unitproject's
excludestill named it, so exactly one file falls out of every tier.Mutation proven on disk by counting the removed and injected text separately
(
include: INTEGRATION_FILES,1 to 0;INTEGRATION_FILES.slice(1)0 to 1), notby an editor's exit code. Pin exit 1, on two cases:
vitest run collects a different population than the filesystem holds: expected [ …(229) ] to deeply equal [ …(230) ]files the predicate calls integration that vitest did NOT collect into that projectWorth stating precisely: the silent-skip is caught by the disk-versus-collected
case, not by the union case — the root run is the union of the projects, so a
file in no project disappears from both sides of that comparison and only the
filesystem walk still sees it. That is the case doing the load-bearing work, and
it survives this change untouched.
Direction 2 is the one that matters for accepting this. Queue-safety is
trivial to buy by weakening a pin, and that is exactly what must not have
happened here: a file matching NO project still reds, on the case that actually
detects the silent skip. The pin is not skipped, not quarantined, not softened,
and
INTEGRATION_FILESstill exists and still drives both projects — it iscomputed instead of typed.
Restored by blob identity —
git hash-objectequal to theHEADblob(
cbadc4ce…),git diff HEADempty,git status --porcelainempty — under atrap … EXIT INT TERMon absolute paths. Control re-run on the restored tree:green, 22 tests.
No rebuild leg is owed, and that is a property of how the pin imports its
subject, not a convenience: the pin reaches the predicate through a relative
source path that Vite transforms from
srctext, andvitest listreadsvitest.config.tsas source. Nothing on either path resolves through apackage's
exportsintodist/, so there is no artifact whose staleness couldgreen a mutated leg.
What a pin is still for, once the list is derived
Deriving deletes the copy — and with it the accidental second opinion the copy
gave, because config and pin now compute the same answer from the same code and
will always agree. That independence is replaced in kind, not dropped:
unitandintegrationpartition every test fileon disk, read from vitest's own resolution. This is the defect the split can
cause — vitest 4.1.10 silently skips a file matching no project and reports
the whole-suite run green — and the reason the list can never simply go away.
integrationversus an independent re-derivation. Entries are handed toinclude/excludeas globs, so a path the walk spells one way and theglobber reads another lands in the wrong tier while every count still looks
right. Unlike the old comparison, it cannot fire because a qualifying file
arrived on
main.false positive the predicate was tuned against (type-only imports, spelling
lists, comment-only prose,
child_processwith no entry point) — plus a unioncheck that every declared signal has a fixture. Without these, a predicate
that matched nothing would empty the integration tier, serialise the suite
back into
unit, and leave every population assertion green.list again.
The fixtures live outside
test/on purpose. The predicate reads sources astext and
maskCommentsleaves string literals intact, so a fixture spelling aspawn would classify its own host file the moment that host is a
*.test.ts— the pin would move itself into the integration tier and fail itsown last case. The module says so where someone would be tempted to move it.
Cost
The derivation reads and masks all 230 test files once per config load: ~0.42s
cold / ~0.27s warm on this box, of which
maskCommentsis ~0.38s and theregexes ~0.01s. A raw-text pre-filter would remove most of it and is
deliberately not taken — it would add a second hand-maintained token list
that must track the predicate, which is the exact class of copy this PR deletes.
Verification
Gate union derived on the final tree with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands(fresh tree, no staleness warning), exit captured before any pipe.
node scripts/check-test-completeness.mjs— NOT MEASURED, exit 3, in thegate's own words:
PREREQUISITE NOT MET — this gate grades a savedturbo run testlog, and no log was named.…There is no local log to hand it, so the local reading for this gate is NOT MEASURED.⛔ It is not a red, and there is nothing here to fix.pnpm check:dual-build-cjs-loadsandpnpm check:type-check-debtfirstreturned the same prerequisite verdict (
this gate reads built output); bothexit 0 after the build below, so they are measured, not assumed.
turbo run build --filter='./packages/*' --filter='./packages/*/*'—71 successful, 71 total.
vitest run --project unit --maxWorkers=2— 158 test files,2092 tests, all passed, 288s. Exit captured before any pipe.
vitest run --project integrationover all 72 files, runin three shards under one hold of the shared verify lock (
VERDICT command-exit 0, held 29m06s). All three shards green — 21 + 27 + 24 = 72 test files,158 + 289 + 120 = 567 tests, all passed, zero
FAILlines across the threelogs. Sharding is not a narrowing: the three shards are a partition of the
same 72 files the tier collects, and the runner banks each green shard so a
container cap kill cannot cost completed work.
pnpm lint(eslint . --no-inline-config) — exit 0,no problems reported. Declared: this one ran OUTSIDE the verify lock, after
two 9-minute queue timeouts, on the same basis as the 27 gate scripts above —
the lock's own status text names
check:*gate scripts as unlocked siblingwork it never excludes, and an eslint scan is that class rather than a build
or a suite.
Every build and test run went through
scripts/pm/os-verify-lock.sh, verdictread from its
VERDICTline, never from a bare$?. The lock was heavilycontended (measured holds of 15, 21 and 29 minutes; six queue timeouts on this
card), which is why the integration tier is sharded and resumable.
pnpm --filter @objectstack/cli typecheckexits 0 and that verdict isempty for this diff. The package's
tsconfig.jsonisinclude: ["src"], sotsc --listFilesreturns 0 hits for all four files in this PR. It is a truesentence that says nothing about the change; recorded as NOT MEASURED rather
than claimed as coverage. Pre-existing and filed separately as #14710, not
repaired here.
Why this is worth more than one anchor
While this was in progress the sibling anchor for
test/run-dev-unbuilt-workspace.e2e.test.ts(card #14648, a different file andnot this PR's scope) ejected another of this seat's PRs from the merge queue and
now stands at five independent hits. That is the second merge-queue eviction
this round caused by an instrument rather than by the PR under test — the
class of failure this change closes for the partition pin. The pattern worth
naming: a check that reads the whole tree, runs inside the queue, and compares
it against something frozen at branch-head time will evict bystanders, and the
durable repair is to stop freezing rather than to soften the check.
No changeset
Nothing this PR touches is published:
packages/clishipsfiles: ["dist", …],and the diff is a test-harness config plus two modules only that config and the
pin import, plus the pin.
pnpm check:published-filesis green. The PR carriesskip-changesetfor that reason.🤖 Generated with Claude Code
https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza