Skip to content

[finding] scripts/sync-template-versions.mjs has no --self-test and runs nowhere in CI — the version-time rewriter is only ever executed on a release PR that CI does not trigger #9348

Description

@os-zhuang

Found while implementing #9264 (extending that script to stamp the template manifest's specVersion). Out of that card's scope — it moves package.json and .github/workflows/lint.yml, neither in #9264's declared file surface. Filed unassigned.

The observation

scripts/sync-template-versions.mjs runs in exactly one place — the root version script:

"version": "changeset version && node scripts/sync-protocol-version.mjs && node scripts/sync-template-versions.mjs && node scripts/sync-docs-image-tags.mjs",

changesets/action calls pnpm run version when preparing the release PR, and that PR is opened with the default GITHUB_TOKEN, so it does not trigger CI. The script's own header says exactly this, as the reason it must exist at all:

release PRs opened by changesets/action with the default GITHUB_TOKEN do not trigger CI, so fixing the files at version time is the only spot that cannot be skipped.

⇒ The rewriter's logic is executed by no gate, ever. What CI checks is only its committed output, via the ratchets in packages/create-objectstack/src/template-consistency.test.ts. Those catch a stale committed value; they cannot catch a rewriter that has stopped rewriting, because on a green corpus a working rewriter and a broken one produce identical (empty) results.

Why this is the exact class the repo already solved once

scripts/sync-docs-image-tags.mjs is the sibling case and it was settled in #9064 by giving the rewriter a --self-test and wiring it into lint.yml. That workflow's comment states the principle better than this issue can:

Only the --self-test runs here, and that is the whole point: the rewriter itself has nothing to do on a green corpus, so CI can never observe it working. The self-test is where a STALE fixture is observed going green through the gate's own checkSurfaces, and — the control that matters just as much — where a CLEAN fixture is observed left byte-identical and unwritten, because an over-eager rewriter would silently corrupt [the documented scheme].

sync-template-versions.mjs is the same shape and did not receive the same treatment. Related prior art, all closed, all this class: #8110, #8170, #6923, #6509.

Why it is worth doing now specifically

#9264 has just grown this script from two hard-coded file paths into a discovery walk with a table of stamps and a collected-problems failure path — more logic, and logic whose failure modes (silently skipping a template, matching zero surfaces and reporting success) are precisely the ones that a green corpus cannot reveal. The reverse-verification for that card had to be performed by hand, by mutating the tree and reading the two scripts' output side by side. A --self-test is that verification made repeatable.

Suggested shape (not a ruling)

Mirror sync-docs-image-tags.mjs:

  • add --self-test to scripts/sync-template-versions.mjs, exercising both directions over temp fixtures: a STALE template observed being rewritten, a CLEAN one observed left byte-identical and unwritten, a template with a MISSING stamp observed exiting 1, a second template observed being covered (the discovery walk), and the zero-template vacuous-green guard;
  • add a check:template-version-sync root script and a step in lint.yml, alongside check:docs-image-tag-sync.

Dedupe

Searched open issues for sync-template-versions, "version-time rewriter self-test", and the self-test/no-CI class. The class has several closed members (above); nothing open covers this script.

Refs: #9264 (the card this came out of), #9064 (the sibling rewriter's self-test), #2907 (the drift class the script exists for).


Generated by Claude Code

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions