Uh oh!
There was an error while loading. Please reload this page.
ci(lint): run the skill-frame freshness gate's self-test half in CI (#8110) - #8171
Merged
Merged
Conversation
…8110) `check:skill-frame-freshness` is `--self-test && the scan`, and only the SCAN half is CI-hostile: on a fresh merge ref "is this tree current with origin/main" is a tautological green, and on any PR that legitimately edits the decision frame it is a false red. That reasoning — the script's own header — was keeping the OTHER half out of CI too, and the price is measured on #8024: the self-test fixture had stranded twice, neither time noticed, while the gate sat in the gate list reading as coverage and protecting nothing. Add one step to lint.yml's `typecheck` job, next to `check:skill-frame-sync`, running only `node scripts/check-skill-frame-freshness.mjs --self-test`. Invoked with `node`, deliberately not `pnpm check:skill-frame-freshness`, which would drag the scan half in with it. Same step shape as the changeset-family self-tests already in this job (#6509): self-test halves in lint.yml, the CI-hostile scan left where it belongs. Measured before wiring, in a `--depth 1` clone with no remote-tracking ref: green, ~1.3s, no network and no history — the specimen is manufactured from the tree's own frame documents and driven through temp git repos whose `origin` points at a path that does not exist. Adversarially: rewording axis prose across all three frame files keeps it green; a structural change (an axis dropped, or a declaring sentence reworded past the shared anchors) reddens it AND reddens `check:skill-frame-sync` above with the same diagnosis, so this step adds no failure class the neighbouring step does not already have. Claude-Session: https://claude.ai/code/session_01P6QzoCR4U6RKxdbCHMJp3V Co-authored-by: Claude <noreply@anthropic.com>
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
hotlong
marked this pull request as ready for review
August 12, 2026 19:16
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#8110
One step added to
lint.yml'stypecheckjob, immediately aftercheck:skill-frame-sync:Invoked with
node, deliberately notpnpm check:skill-frame-freshness— that npm script is--self-test && the scan, and the scan half is the CI-hostile one the script's own header argues against (tautological green on a merge ref, false red on any PR that legitimately edits the decision frame). Wiring the npm script would recreate exactly the problem this card exists to route around. No scan-half wiring was added, here or anywhere.The step shape has precedent in the same job: the changeset-family self-tests (#6509) are already in
lint.ymlas "the SELF-TEST halves only — the real scans stay inpr-automation.yml'schangeset-check, and the split is the whole point of this step". This is the second instance of that split, so the step name says self-test half only and the comment block says why, in order that the omission reads as a decision rather than an oversight.Measurements
The dispatching PM held four hypotheses weakly and asked for readings rather than agreement. All four were measured. Three held; the fourth needs a correction to how it was stated.
A. Hermetic — confirmed, and no checkout change is needed. The self-test manufactures its specimen from the working tree's own frame documents and drives it through temp git repos whose
originpoints at a path that does not exist, so the offline rung is exercised by real code without touching the network. Measured in agit clone --depth 1of the branch, which carries one commit and norefs/remotes/origin/mainat all: 12/12 cases green. It is also moot for the placement chosen — thetypecheckjob already checks out withfetch-depth: 0for the authorable-surface deletion gate — but the depth-1 reading is the one that proves the step would survive a future job that does not.B. Cheap — confirmed. ~1.3s in the depth-1 clone, ~2.0s in a full worktree, wall-clock, including node start-up. It needs no
node_modules(the script imports only node builtins andcheck-skill-frame-sync.mjs), so it is cheaper than any of its neighbours, all of which pay a pnpm script hop.C. Non-empty family — confirmed, with exactly one other member, filed as #8170 and not fixed here per the dispatch ruling.
check:dev-prereqsis the only other gate matching the criterion (a scan half that cannot run in CI, keeping a CI-safe--self-testhalf out with it): its self-test is green in 0.12s on an unbuilt checkout with no dependencies installed, while its scan is hard-red there ("The workspace is not built — 67 of 67 workspace packages..."). Enumeration method, and the two rejected candidates with reasons, are written up in #8170 so the negative half of the survey is reviewable rather than asserted. One methodological note worth carrying: workflow YAML comments must be stripped before matching, orcheck:platform-checklistlands in the "wired" column on the strength of alint.ymlcomment explaining why it is deliberately not wired.D. Frame-editing PRs stay green — confirmed for wording, but the claim as stated was too broad; here is the sharper version. A structure-preserving edit does not redden it: rewording an axis body in
.claude/agents/os-dev.md, and rewording axis prose across all three frame files at once, both left the self-test andcheck:skill-frame-syncgreen. What does redden it is a structural change — an axis added or removed, or a declaring sentence reworded past the shared anchors. Measured that case too: rewordingos-dev.md's declaring sentence givesand, in the same tree,
node scripts/check-skill-frame-sync.mjsexits 1 with the same diagnosis. So the new step adds no failure class the step directly above it does not already have: any input that reddens it already reddenscheck:skill-frame-sync, which has been required in this job since #5865, and the self-test's own message names that gate as the thing to fix first. That is the honest form of D's premise — not "nothing can redden it", but "nothing can redden it alone".The one real cost is on a PR that deliberately restructures the frame (adds a fourth axis, drops one): it must update the derivation in
check-skill-frame-freshness.mjsin the same PR. That PR was already obliged to updatecheck-skill-frame-sync.mjs's anchors, so the obligation is not new — the CI step only makes it arrive before merge instead of after.Verification
node scripts/check-skill-frame-freshness.mjs --self-test— 12/12 cases pass, on the worktree and in the depth-1 clone.node scripts/pm/dispatch-gates.mjs .github/workflows/lint.yml) and run:check:changeset-gate-self-tests,check:node-version,check:nul-bytes(7422 files scanned, 0 control bytes),check:required-contexts,check:shard-attestation,check:type-check-coverage,check:workflow-status-functions,node scripts/check-changeset-no-major.mjs— all green.check:type-check-debtwas not run: its--re-measurehalf refuses to run without the full workspace built ("55 workspace dependencies of the ledgered packages have no built type entry point on disk"), and building 77 packages to re-measure TypeScript debt for a change that touches no TypeScript source is not proportionate. Its--self-testhalf is shared withcheck:type-check-coverage, which passed. CI runs it against a built tree.typecheck; the new one runsnode scripts/check-skill-frame-freshness.mjs --self-test).Changeset
None, by route 2 —
skip-changeset, applied to this PR.lint.yml's own prose prescribes it: "this PR edits a CI-internal script is the textbookskip-changesetcase — such a PR releases nothing". Nothing user-visible ships from a workflow step. Note the interaction the same comment block documents: askip-changesetPR is exempted fromchangeset-checkwholesale, which is exactly why the changeset-family self-tests were moved intolint.ymlunconditionally — and the same reasoning is what puts this step there rather than anywhere gated.Generated by Claude Code