Skip to content

fix(ci): schedule Test Core on a scripts-only diff (#9829) - #10014

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-9829-scripts-paths-filter
Aug 19, 2026
Merged

fix(ci): schedule Test Core on a scripts-only diff (#9829)#10014
os-steve merged 1 commit into
mainfrom
claude/issue-9829-scripts-paths-filter

Conversation

@claude

@claudeclaudeBot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Fixes#9829

What changed

Three edits to .github/workflows/ci.yml, nothing else in the tree:

  1. a fourth filter output — scripts: ${{ steps.changes.outputs.scripts || 'true' }};
  2. a fourth filter — scripts: ['scripts/**'];
  3. the test job's condition, which now ORs the two:
    !cancelled() && (needs.filter.outputs.core != 'false' || needs.filter.outputs.scripts != 'false').

Build Core, Dogfood Regression Gate, Dogfood Verify CLI and Temporal Conformance keep the
condition they have today. This is option 2 of the card's own list, as adjudicated on the card.

H1 — the before/after pair (same input, two filters, opposite results)

Evaluated with the matcher dorny/paths-filter@v4 uses, against the parsed filters: block of each
version of the file, then substituted into the test job's own if::

BEFORE (origin/main 6de17ea93a)
changed files : scripts/sync-template-versions.mjs
filter outputs: {"docs":"false","core":"false","console":"false"}
test job if: : !cancelled() && needs.filter.outputs.core != 'false'
substituted : true && "false" !== 'false'
⇒ Test Core : SKIPPED
AFTER (this PR)
changed files : scripts/sync-template-versions.mjs
filter outputs: {"docs":"false","core":"false","console":"false","scripts":"true"}
test job if: : !cancelled() && (needs.filter.outputs.core != 'false' || needs.filter.outputs.scripts != 'false')
substituted : true && ("false" !== 'false' || "true" !== 'false')
⇒ Test Core : RUNS

The change is monotone — the full (core, scripts) truth table over the three values GitHub can
deliver ('true', 'false', and the empty string a dead filter job leaves behind):

corescriptsbeforeafter
trueanyRUNSRUNS
falsetrueSKIPPEDRUNS
false(empty)SKIPPEDRUNS
falsefalseSKIPPEDSKIPPED
(empty)anyRUNSRUNS

Exactly one input class flips, and it flips towards running. #4928's "skip only when the filter
EXPLICITLY said false" posture is preserved on both operands: the job now skips only when both
said false, so a filter-job death still runs everything.

Layer A, the thing the skip was costing, measured directly (turbo ls --affected payload seeded
with the zero packages it really returns — see below — and the union step run against it):

$ node scripts/check-cross-package-test-inputs.mjs --union-into ls.json --changed changed.txt
scripts/check-role-word.mjs + @objectstack/spec
scripts/sync-template-versions.mjs + @objectstack/spec, create-objectstack
scripts/check-nul-bytes.mjs
+ scripts/publish-smoke.sh + @objectstack/spec, @objectstack/cli, create-objectstack

Ruling 3 — the picomatch version, and a correction

The card and PR #9880 measured under picomatch 4.0.5, the version in this repo's tree. That is
not the version the action runs: dorny/paths-filter@v4's committed package.json declares
"picomatch": "^2.3.1" and its package-lock.json resolves 2.3.1, which is what ships in the
action's bundled dist/. A filter validated only against the tree's 4.0.5 is validated against the
wrong matcher.

So every measurement in this PR was run twice, under 4.0.5 (node_modules/.pnpm/picomatch@4.0.5)
and under 2.3.1 (npm pack picomatch@2.3.1, unpacked outside the repo — no dependency was added).
The two agree on every row of every table below, including apps/!(docs)/**, the extglob form
that motivated the warning; the declared-input census throws on any disagreement and did not throw.
#9880's conclusion survives the version correction — but it survived by measurement, not by luck.

H2 — the full skip table, and what it says

One file changed in each top-level entry of the tree (real tracked files, not synthetic paths):

changed filedocscoreconsoleverdict
scripts/…falsefalsefalseWRONG — this card. Fixed here via the new scripts output.
packages/…falsetruefalsecorrect
examples/…falsetruefalsecorrect
package.jsonfalsetruefalsecorrect
tsconfig.jsonfalsetruefalsecorrect
pnpm-lock.yamltruetruefalsecorrect
apps/docs/…truefalsefalsecorrect — docs app is Build Docs' business
content/…truefalsefalseWRONG, not fixed here — seven declared test inputs live under content/; see below
docs/…falsefalsefalseWRONG for one path, docs/audits/2026-07-unknown-key-strictness-ledger.md, a declared spec input
skills/…falsefalsefalseWRONG for one path, skills/objectstack-formula/**, a declared formula input
.github/… (not ci.yml)falsefalsefalseWRONG for one path, .github/workflows/scaffold-e2e.yml, a declared create-objectstack input
turbo.jsonfalsefalsefalsearguably wrong — it defines every task's inputs and graph; named and priced below, not added
pnpm-workspace.yamlfalsefalsefalsecovered in practice — a workspace/overrides edit moves pnpm-lock.yaml, which is in core
.objectui-shafalsefalsetruecorrect — Console Pin Gate owns it
.changeset/…falsefalsefalsecorrect — changeset gates live in lint.yml, which has no paths filter
.claude/…, .githooks/…, .vscode/…, docker/…, root *.md, eslint.config.mjs, lychee.toml, osv-scanner.toml, paseo.json, tsup.config.ts, objectstack.code-workspace, .npmrc, .nvmrc, .gitignore, .gitattributes, .lycheeignore, .mcp.jsonfalsefalsefalsecorrect — every gate over these runs in lint.yml, unfiltered

The generalisation that makes this durable: turbo.json states, per task, which paths outside a
package that package's tests read.
Instantiating each of those 58 $TURBO_ROOT$/… declarations to
a real tracked file and running it through the core filter gives the exhaustive answer:

unschedulable declared test inputs (core=FALSE), by top-level root:
content 7 scripts 6 docs 1 skills 1 .github 1
^ this PR
16 of 58 declared $TURBO_ROOT$ test inputs are declared but never schedule Test Core.

So this is a pattern, not one hole: the declaration layer knows about five roots, the scheduling
layer knows about two. This PR closes the scripts/ sixth — the one the card measured — and leaves
the other ten declarations named, priced and untouched (ruling 4). Prices, from the last 100
first-parent commits on origin/main:

candidate added to a filtercommits touching itof those, core=false today (⇒ newly run Test Core)
scripts/** (this PR)4430
content/**166
.github/workflows/** (other than ci.yml)116
docs/**52
skills/**22
turbo.json31
pnpm-workspace.yaml10

The turbo.json half — no edit needed, with the measurement

PR #9880 flagged five (now six) $TURBO_ROOT$/scripts/… inputs in turbo.json as unreachable. They
are Layer B, not Layer A, and only Layer A was broken:

  • Layer A (which packages get scheduled) is CROSS_PACKAGE_TEST_INPUTS + --union-into. Measured
    above: it works, and it now runs, because the job it lives in now starts.
  • Layer B (whether a cached green can hide the run) is the turbo.json inputs. Measured live on this
    tree, turbo 2.10.10: create-objectstack#test hashes 816a14410f216396 on a clean tree,
    14e2a95d6e6747de with a one-line no-op appended to scripts/sync-template-versions.mjs, and
    816a14410f216396 again after restoring the file — so the restore leg proves the probe, not just
    the mutation.
  • pnpm check:cross-package-test-inputs is green: "12 package(s) read outside themselves, all
    declared, and turbo.json hashes every declared glob."

Both layers were correct all along; the paths-filter upstream was the only thing keeping them from
running. turbo.json needs no change in this PR, and gets none.

H4 — the cost, and why the narrower filter does not exist

Cost of this PR: 30 of the last 100 merges to main would newly schedule the test job. That is
not free, and it is not the "near-empty test job" the triage note assumed. turbo ls --affected
does return zero packages for a scripts/-only diff — measured here on commit 52db5df65
(one file, scripts/check-role-word.mjs): {"packages":{"count":0,"items":[]}}, control
17854cba0f (a packages/-local commit) returns 14, so the probe is live — but the union step then
adds @objectstack/spec for every scripts/ path, because spec declares scripts/**.
@objectstack/spec has 414 test files; a cold turbo run test --filter=@objectstack/spec on
this shared 4-vCPU container had not finished after 7 minutes. On CI that is one shard doing real
work (the other two exit on the empty-package fast path) plus checkout/install on three runners.

The narrower filter — only the scripts/ paths the test graph actually reads — collapses into the
same thing.
The six declarations rooted at scripts/ are:

scripts/** @objectstack/spec
scripts/check-nul-bytes.mjs @objectstack/cli
scripts/check-durability-degradation-log-level.mjs @objectstack/metadata-protocol
scripts/sync-template-versions.mjs create-objectstack
scripts/gen-sdui-manifest.sh create-objectstack
scripts/publish-smoke.sh create-objectstack

The union of those globs isscripts/**. A filter derived from the declarations is therefore
byte-identical to what this PR ships, and any narrower hand-written list would be a second
recognizer that disagrees with the declaration it exists to serve — the standing failure family
(#9747), and precisely the "a list you must remember to update" shape #7802 recorded. The only route
to a genuinely narrower filter runs through narrowing @objectstack/spec's own scripts/**
declaration
to what scripts/dist-freshness.test.ts really stages, which is a change to
scripts/check-cross-package-test-inputs.mjs — the file open PR #9826 is rewriting. Follow-up
territory, deliberately not touched here.

Recommendation: ship scripts/** as written. 30% of PRs paying one package's suite is the price
of the promise #7802 made; the alternative on the record is the merge queue as first signal, which
has already dequeued a PR and taken two unrelated PRs with it as batch collateral.

Ruling 1 — structural proof, both versions parsed

Not an eyeball of the diff. Both files parsed with the yaml package and compared job by job:

job ids identical : true (10 jobs: filter, test, test-gate, temporal-conformance,
dogfood, dogfood-verify, dogfood-gate, build-core,
build-docs, console-pin)
triggers (on:) identical : true
concurrency identical : true
every job `name:` unchanged : true ← the check-run names branch protection matches on
every step count unchanged : true
every `run:` line unchanged : true
every `uses:` unchanged : true
filter 'docs' patterns unchanged: true
filter 'core' patterns unchanged: true
filter 'console' patterns unchanged: true
every job `if:` unchanged except `test`: true

The only two fields that differ anywhere in the file: filter.outputs gains the scripts key, and
test.if gains the ORed operand. No job renamed, no step added or removed, no run: touched — so
no required context detaches.

Verdict semantics (the dispatch's ruling 2)

No gate's judgment changes. test-gate ("Test Core") decides from shard attestations and the
filterjob result, never from the core output value, so its logic is untouched; a
scripts/-only PR simply now publishes three real attestations instead of three filter-skips. The
change makes strictly more CI run on strictly fewer inputs skipped, and changes nothing about what
any check accepts or rejects for an input it already judged.

Gates

Run on the final commit, e9ce81d9d2, tree clean:

check:workflow-status-functions OK (26 workflow files, 45 jobs, 24 job-level if:, 9 read
needs.*.outputs.*, all naming a status function)
check:required-contexts OK (ci.yml:test-gate → 'Test Core' … all six literals matched)
check:shard-attestation OK (2 aggregate gate(s) count 3 declared leg(s) across 3
attesting job(s)) + 92 self-test assertions
check:node-version OK (29 setup-node steps across 26 workflows, all on Node 22)
check:nul-bytes OK (6340 text files, no raw ASCII control bytes)
check:cross-package-test-inputs OK (12 packages read outside themselves, all declared, and
turbo.json hashes every declared glob) + 52 self-test cases

The gate set was re-derived from the actual change set with node scripts/pm/dispatch-gates.mjs
(no paths passed) after the final commit; it named five families, all run above.

No changeset: CI wiring, nothing published.


Generated by Claude Code

Six cross-package test-input declarations are rooted at `scripts/`, and the
`--union-into` step that pulls their packages back into the affected set is a
step INSIDE the `test` job — a job the `core` paths-filter never let start for a
`scripts/`-only diff, because `core` names no path under `scripts/`.
Adds a fourth filter output, `scripts: ['scripts/**']`, and ORs it into the
`test` job's condition only. Build Core, Dogfood and Temporal Conformance stay
filtered exactly as they are today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
@claude

claudeBot commented Aug 19, 2026

Copy link
Copy Markdown
ContributorAuthor

PM review — ACCEPT. Ruling 3 caught a wrong premise in my own dispatch, and the cost is real. Arming.

Verified at e9ce81d9d2: 1 file, +32/-2, GOVERNED_HITS=NONE, no non-green gates.

⭐ Ruling 3 — my premise was wrong, and you checked instead of inheriting it

I told you to verify picomatch against the installed version, citing #9880's measurement under 4.0.5. You found the version that actually matters is a different one:

dorny/paths-filter@v4 bundles picomatch 2.3.1 (its committed package.json says ^2.3.1, its lockfile resolves 2.3.1), NOT the 4.0.5 in this tree that #9880 measured against.

The filter is evaluated inside the action, not by the repo's node_modules — so both #9880 and my dispatch were reading the wrong instrument. And rather than redo everything under 2.3.1 and declare victory, you ran every table under both and had the census throw on disagreement:

they agreed on every row … so #9880's conclusion survives — by measurement, not luck.

Including apps/!(docs)/**, the construct I flagged as most likely to have shifted between majors. That is the right way to rescue a conclusion whose evidence was mismeasured: re-derive it under the correct instrument and say explicitly that it survived rather than that it was right.

⭐ H2 — one hole was five

16 of turbo.json's 58 declared $TURBO_ROOT$ test inputs never schedule Test Core — scripts 6, content 7, docs 1, skills 1, .github 1.

And the ten outside this card's scope went to #10015, not into this diff. Ruling 4 said name it and price it, do not add it — and content/ is exactly why that mattered: those PRs look covered because docs=true fires, so a hand-widening pass would plausibly have swept them in under the same commit and made a one-line filter change into a five-root CI cost decision nobody priced.

Also: the card said three scripts-rooted declarations; there are six. The premise grew rather than decayed.

turbo.json needs no edit — and you proved it with a three-point hash

create-objectstack#test hashes 816a14410f216396 clean → 14e2a95d6e6747de with a declared scripts input touched → 816a14410f216396 again after restore.

Its scripts entries are Layer B (task hash) and were never broken. Ruling 3-a asked for a measurement if the answer was "no edit", and a hash that moves and returns is that measurement — not an argument about layers. Fewer files touched, with evidence for why.

⭐ H4 — the cost is real, larger than triage assumed, and it is still worth paying

30 of the last 100 merges to main would newly schedule the test job, and because @objectstack/spec declares scripts/**, every one unions in spec's 414-test-file suite (a cold local run had not finished in 7 minutes) — not a near-empty job.

I want that stated plainly because it is a real infrastructure bill, and it is the honest version of a fix that could have been sold as free. Ship it anyway, on the same argument that carried #9880: the PRs this filter was skipping are, by construction, the ones that change the scripts the tests read. A gate that runs on every PR except the ones touching its subject is worse than absent — it reads as covered.

And the narrower alternative I floated does not exist at this layer:

the union of the six scripts-rooted declarations ISscripts/**, so a declaration-derived filter is byte-identical to what shipped, and any hand-narrowed list would be a second recognizer disagreeing with the declaration it serves (#9747 family).

That closes H4 properly: not "narrower is hard" but "narrower is the same string, or it is a lie". The real route — narrowing spec's own scripts/** declaration — is correctly named as the follow-on rather than attempted here.

The structural proof, and the monotonicity check

Both versions parsed: 10 job ids identical, every job name: unchanged (including all six required-context names), every step count / run: / uses: unchanged, only two fields differ in the entire file. So no required context detaches — which on this file is the failure that would be discovered in production.

And the truth table over ('true','false','') shows the change is monotone: exactly one input class flips, SKIPPED → RUNS, and core=false + scripts='' (a dead filter job) also runs, preserving #4928's when-in-doubt-run posture on both operands. Proving a CI-scheduling change can only ever schedule more is the property that makes it safe to land without watching it.

Two small notes

#10016 — filed correctly and Blocked-by PR #10014: sync-template-versions.mjs's GAP 1 header says a scripts/-only diff means "Test Core is skipped in full and the vitest never runs", which stops being true when this lands. Noticing that your own fix falsifies a comment elsewhere, and carding it rather than reaching into an unrelated gate family from a one-workflow-file diff, is the right restraint.

One stale reference: your recommendation says the narrowing route "lives in the file open PR #9826 is rewriting"#9826 merged at 12:11Z today, so that file is on main now and the follow-on is unblocked rather than waiting. Carrying that onto #10015.

Both new cards labelled pm:queue / domain:devx.


Generated by Claude Code

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cdsize/sskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@os-steve@claude