Skip to content

fix(scripts): derive the skill-frame-freshness self-test fixture from the live documents - #8109

Merged
hotlong merged 1 commit into
mainfrom
claude/issue-8024-skill-frame-freshness-fixture
Aug 12, 2026
Merged

fix(scripts): derive the skill-frame-freshness self-test fixture from the live documents#8109
hotlong merged 1 commit into
mainfrom
claude/issue-8024-skill-frame-freshness-fixture

Conversation

@hotlong

Copy link
Copy Markdown
Contributor

Fixes#8024

The defect, and the sharper shape it turned out to have

pnpm check:skill-frame-freshness was red on a clean main, and red in the worst place. package.json runs it as --self-test && (the real scan), so the fixture failure short-circuited the scan entirely:

node scripts/check-skill-frame-freshness.mjs --self-test EXIT=1 (fixture drifted)
node scripts/check-skill-frame-freshness.mjs EXIT=0 frame is current
4 copies / 3 files / 3 axes

The gate was healthy; it was never reached. There was no frame divergence to chase — consistent with check:skill-frame-sync staying green. The skill files are correct post-rewrite; the fixture was stale. Nothing under .claude/skills/, skills/ or .claude/agents/ is touched by this PR.

Worth recording for the risk model: this gate is deliberately not wired into lint.yml (its header explains why — on a fresh CI merge ref it is a tautological green, and on any PR legitimately editing the frame it would be a false red). So nothing in CI could have reported it, and main stayed red until a PM ran it by hand at dispatch time.

Why the fixture, not the wording, is the fix

The fixture was a table of literal find/replace rules that manufactured the historical two-axis specimen by re-spelling the real documents' prose. The 2026-08-12 principles-only rewrite reworded one axis line, the rule matched nothing, and the fixture threw. The sibling check:skill-frame-sync — which parses rather than re-spells — survived the same rewrite untouched. That is the argument for deriving.

The specimen is now located the way the gates already read these documents:

  • the two count sentences — through each copy's own start / binding anchor;
  • the axis entry to drop — through the shared entry parser plus AXIS_MAP;
  • the in-file count mentions — through a shared mention scan.

Nothing in the fixture spells a sentence of the frame, so any rewording, re-indentation or reflow carries it along. A change of structure still fails, loudly, naming the file and line to edit.

check-skill-frame-sync.mjs gains three exports for this (ENTRY_START, axisEntryStarts, frameCountMentions) and its own mention check now runs through the last of them, so the two scripts cannot drift apart about what a mention is. Cases 8 and 9, which broke a declaring sentence by spelling it out, are derived from the same anchors for the same reason.

The tautology question, measured rather than assumed

"Derive the fixture from the live files" is vacuous when it makes a check compare files to themselves. It does not here, and the distinction is which half is derived: what is derived is how the SPECIMEN is manufactured, never what the gate compares. The gate's own comparison — this tree's structure against origin/main — never touches any of it. The self-test still commits the demoted specimen and the real documents as two commits in a temp repo and demands the gate call the older one behind.

Verified by neutering the demotion in two layers, both predicted red before running:

tamperingresult
unmarkEntryLine returns the line unchangedred: un-marking the axis entry left it still matching ENTRY_START at SKILL.md:604
the above, plus the ENTRY_START assertion bypassedred: the demoted copy reads unparseable, expected 2 axes

A demotion that silently did nothing cannot produce a green run.

Reverse verification

Direction predicted first in each case.

Reintroduce the stale fixture (restore origin/main's script). Predicted red on the fixture with the scan never reached — observed exactly that: EXIT=1, Error: self-test fixture drifted, and no scan summary line in the output.

Across the rewrite boundary. The same three documents at ab19075^ (before the principles rewrite) and at HEAD, each run through the old fixture and the new one:

documentshand-copied tablederived (this PR)
pre-rewrite (ab19075^)RED — case 3, mention 三条评估轴GREEN, 12/12
today (HEAD)RED — fixture throwGREEN, 12/12

The top-left cell was not the direction I predicted (I expected green — the wording the table was written against), and it is the most useful result here: the hand-copied table had already stranded before the rewrite, in a second and independent way. Its mention rule spelled 沿三条固定评估轴, which does not cover the bare 三条评估轴 mention in the same file, so its specimen was incoherent and the sync gate reported it — failing the independence case rather than throwing. The gate was red at ab19075^ too, for a different reason than the card diagnoses. Two stranding events in one hand-maintained copy is the evidence for the derived route, not just the aesthetics of it.

I could not date that second stranding: this clone's history is grafted at f7dceed, so git log -S attributes the mention to the graft boundary rather than to a real introducing commit.

Gates

Green locally: pnpm check:skill-frame-freshness (self-test 12/12 and the scan, which now runs at all), pnpm check:skill-frame-sync (12/12 plus its scan), pnpm check:nul-bytes. node scripts/pm/dispatch-gates.mjs on the two changed paths surfaces no additional named family.

eslint was not run locally: no node_modules exists in this container and a full workspace install for two dependency-free root scripts is not a good trade. The applicable rule block for **/*.mjs is no-restricted-imports on @objectstack/spec subpaths, which neither file imports.

No changeset: root gate scripts, nothing released. skip-changeset applied.


Generated by Claude Code

… the live documents
`pnpm check:skill-frame-freshness` was red on a clean `main`, and it was red in
the worst possible place: `package.json` runs `--self-test && <the scan >`, so a
fixture failure short-circuits the real check. The gate stayed in the gate list,
reading as coverage, while scanning nothing — every PR touching the guarded skill
files sailed past it.
The fixture, not the gate, was broken. It was a table of literal find/replace
rules that manufactured the historical two-axis specimen by re-spelling the real
documents' prose; the 2026-08-12 principles-only rewrites reworded one axis line,
the rule matched nothing, and the fixture threw. The skill files were correct
throughout, and the sibling `check:skill-frame-sync` — which parses rather than
re-spells — survived the same rewrite untouched.
So the specimen is now derived the way the gates already read these documents:
count sentences through each copy's own `start`/`binding` anchor, the axis entry
to drop through the shared entry parser plus `AXIS_MAP`, and in-file count
mentions through a shared mention scan. Nothing here spells a sentence of the
frame, so any rewording, re-indentation or reflow carries the fixture with it.
This does not make the gate tautological: what is derived is how the SPECIMEN is
manufactured, never what the gate compares. The self-test still commits the
demoted specimen and the real documents as two commits in a temp repo and demands
the gate call the older one behind — a demotion that silently did nothing fails
loudly in `verifyDemoted()`, which names the file and line to edit.
`check-skill-frame-sync.mjs` gains three exports for this (`ENTRY_START`,
`axisEntryStarts`, `frameCountMentions`) and its mention check now runs through
the last of them, so the two scripts cannot disagree about what a mention is.
Cases 8 and 9, which broke a declaring sentence by spelling it out, are derived
from the same anchors for the same reason.
@vercel

vercelBot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 12, 2026 3:46pm

Request Review

@hotlonghotlong added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 12, 2026 — with Claude
@hotlong
hotlong marked this pull request as ready for review August 12, 2026 16:23
@hotlong
hotlong added this pull request to the merge queueAug 12, 2026
Merged via the queue into main with commit 801d952Aug 12, 2026
24 checks passed
@hotlong
hotlong deleted the claude/issue-8024-skill-frame-freshness-fixture branch August 12, 2026 16:34
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check:skill-frame-freshness is red on main — self-test fixture drifted after the principles-only rewrites, so the gate never scans

2 participants

@hotlong@claude