Uh oh!
There was an error while loading. Please reload this page.
fix(scripts): derive the skill-frame-freshness self-test fixture from the live documents - #8109
Merged
Merged
Conversation
… the live documents `pnpm check:skill-frame-freshness` was red on a clean `main`, and it was red in the worst possible place: `package.json` runs `--self-test && <the scan >`, so a fixture failure short-circuits the real check. The gate stayed in the gate list, reading as coverage, while scanning nothing — every PR touching the guarded skill files sailed past it. The fixture, not the gate, was broken. It was a table of literal find/replace rules that manufactured the historical two-axis specimen by re-spelling the real documents' prose; the 2026-08-12 principles-only rewrites reworded one axis line, the rule matched nothing, and the fixture threw. The skill files were correct throughout, and the sibling `check:skill-frame-sync` — which parses rather than re-spells — survived the same rewrite untouched. So the specimen is now derived the way the gates already read these documents: count sentences through each copy's own `start`/`binding` anchor, the axis entry to drop through the shared entry parser plus `AXIS_MAP`, and in-file count mentions through a shared mention scan. Nothing here spells a sentence of the frame, so any rewording, re-indentation or reflow carries the fixture with it. This does not make the gate tautological: what is derived is how the SPECIMEN is manufactured, never what the gate compares. The self-test still commits the demoted specimen and the real documents as two commits in a temp repo and demands the gate call the older one behind — a demotion that silently did nothing fails loudly in `verifyDemoted()`, which names the file and line to edit. `check-skill-frame-sync.mjs` gains three exports for this (`ENTRY_START`, `axisEntryStarts`, `frameCountMentions`) and its mention check now runs through the last of them, so the two scripts cannot disagree about what a mention is. Cases 8 and 9, which broke a declaring sentence by spelling it out, are derived from the same anchors for the same reason.
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
This was referenced Aug 12, 2026
hotlong
marked this pull request as ready for review
August 12, 2026 16:23
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#8024
The defect, and the sharper shape it turned out to have
pnpm check:skill-frame-freshnesswas red on a cleanmain, and red in the worst place.package.jsonruns it as--self-test && (the real scan), so the fixture failure short-circuited the scan entirely:The gate was healthy; it was never reached. There was no frame divergence to chase — consistent with
check:skill-frame-syncstaying green. The skill files are correct post-rewrite; the fixture was stale. Nothing under.claude/skills/,skills/or.claude/agents/is touched by this PR.Worth recording for the risk model: this gate is deliberately not wired into
lint.yml(its header explains why — on a fresh CI merge ref it is a tautological green, and on any PR legitimately editing the frame it would be a false red). So nothing in CI could have reported it, andmainstayed red until a PM ran it by hand at dispatch time.Why the fixture, not the wording, is the fix
The fixture was a table of literal find/replace rules that manufactured the historical two-axis specimen by re-spelling the real documents' prose. The 2026-08-12 principles-only rewrite reworded one axis line, the rule matched nothing, and the fixture threw. The sibling
check:skill-frame-sync— which parses rather than re-spells — survived the same rewrite untouched. That is the argument for deriving.The specimen is now located the way the gates already read these documents:
start/bindinganchor;AXIS_MAP;Nothing in the fixture spells a sentence of the frame, so any rewording, re-indentation or reflow carries it along. A change of structure still fails, loudly, naming the file and line to edit.
check-skill-frame-sync.mjsgains three exports for this (ENTRY_START,axisEntryStarts,frameCountMentions) and its own mention check now runs through the last of them, so the two scripts cannot drift apart about what a mention is. Cases 8 and 9, which broke a declaring sentence by spelling it out, are derived from the same anchors for the same reason.The tautology question, measured rather than assumed
"Derive the fixture from the live files" is vacuous when it makes a check compare files to themselves. It does not here, and the distinction is which half is derived: what is derived is how the SPECIMEN is manufactured, never what the gate compares. The gate's own comparison — this tree's structure against
origin/main— never touches any of it. The self-test still commits the demoted specimen and the real documents as two commits in a temp repo and demands the gate call the older one behind.Verified by neutering the demotion in two layers, both predicted red before running:
unmarkEntryLinereturns the line unchangedun-marking the axis entry left it still matching ENTRY_STARTatSKILL.md:604ENTRY_STARTassertion bypassedthe demoted copy reads unparseable, expected 2 axesA demotion that silently did nothing cannot produce a green run.
Reverse verification
Direction predicted first in each case.
Reintroduce the stale fixture (restore
origin/main's script). Predicted red on the fixture with the scan never reached — observed exactly that:EXIT=1,Error: self-test fixture drifted, and no scan summary line in the output.Across the rewrite boundary. The same three documents at
ab19075^(before the principles rewrite) and atHEAD, each run through the old fixture and the new one:ab19075^)三条评估轴HEAD)The top-left cell was not the direction I predicted (I expected green — the wording the table was written against), and it is the most useful result here: the hand-copied table had already stranded before the rewrite, in a second and independent way. Its mention rule spelled
沿三条固定评估轴, which does not cover the bare三条评估轴mention in the same file, so its specimen was incoherent and the sync gate reported it — failing the independence case rather than throwing. The gate was red atab19075^too, for a different reason than the card diagnoses. Two stranding events in one hand-maintained copy is the evidence for the derived route, not just the aesthetics of it.I could not date that second stranding: this clone's history is grafted at
f7dceed, sogit log -Sattributes the mention to the graft boundary rather than to a real introducing commit.Gates
Green locally:
pnpm check:skill-frame-freshness(self-test 12/12 and the scan, which now runs at all),pnpm check:skill-frame-sync(12/12 plus its scan),pnpm check:nul-bytes.node scripts/pm/dispatch-gates.mjson the two changed paths surfaces no additional named family.eslintwas not run locally: nonode_modulesexists in this container and a full workspace install for two dependency-free root scripts is not a good trade. The applicable rule block for**/*.mjsisno-restricted-importson@objectstack/specsubpaths, which neither file imports.No changeset: root gate scripts, nothing released.
skip-changesetapplied.Generated by Claude Code