ci(release): validate the post-version tree before the version PR is refreshed

[yinlianghui-tw]

Part of #5397

Maintainer ruling 2026-08-22 (「接受所有」): option A — validate the post-version tree inside changeset-release.yml, after the version step and before changesets/action opens or updates the release PR. The ruling attached a binding pre-condition — price it first — and the price is what chose the implementation shape below.

The measurement came first, and it changed the answer

Re-measured today (the card asked for this explicitly). The defect still holds and is worse than filed: ci.yml now has 849 runs on changeset-release/main (736 when the card was filed), and every recent one is conclusion: action_required with created_at == run_started_at == updated_at — created and immediately parked, nothing executed. Most recent: 2026-08-25T00:12:42Z, run 32792689033.

But the cost premise in the card expired two days ago. The card (2026-08-20) and the dispatch order both price naive option A against a refresh that runs on every push to main — ~18/day × 40 min ≈ 12 h of daily runner time. objectstack#10850 moved the refresh onto a 6-hourly cron, and the Actions API shows the changeover precisely: refresh runs were many-per-hour through 2026-08-23T05:39Z, and from 2026-08-23T06:10Z they are exactly 6 hours apart. Measured: 8 schedule runs in 42 h = 4.0/day, not 18.

	measured
refresh cadence today	4/day (cron 0 */6 * * *)
refresh lane runner time today	106 s (run 32792567720: lane 6 s + release 100 s; install 13 s, build 37 s, the changesets step incl. pnpm changeset:version13 s)
naive A (full pnpm test post-version)	+~40 min/refresh → ~2.7 h/day
this PR (scoped A)	+~68 s/refresh → ~4.5 min/day

ci.yml's own instrumented figure for the unsharded suite is 39 min 51 s of tests / 40 min 19 s of job, recorded in its header.

Why naive option A is the wrong shape — the version step cannot move a source byte

Rendered against this repository's real tree (328 pending changesets, 17.6.0 → 17.7.0), pnpm changeset:version touches 411 paths, and all of them are one of four kinds:

count	path	what moves
330	.changeset/*.md	deleted (consumed by the version)
40	*/package.json	the "version" key and nothing else — 80 changed lines, every one of them "version":
40	*/CHANGELOG.md	generated from the changeset bodies
1	QUICK_REFERENCE.md	rewritten by changeset:version's sync-quick-reference-release.mjs

Zero .ts, zero .tsx, zero config, zero dependency ranges. The source in the post-version tree is byte-identical to the main commit the job checked out — the commit ci.yml's push lane already ran the whole suite over, under coverage, across four shards (~13.8 min wall, median of 30 recent runs). A pnpm test here would re-test bytes tested minutes earlier and report nothing new, at 40 minutes a go, four times a day. That is the cost objectstack#10850 was closed to remove, walking back in through a different door — exactly what the dispatch order forbade shipping unmeasured.

So the validation is scoped to what the diff can actually move, and the union — ci.yml on the source, these steps on the version outputs — is the whole post-version tree.

What the refresh lane now does

Three steps, all scoped to schedule / workflow_dispatch, inserted immediately before Refresh the version PR:

1. Render the post-version tree — pnpm changeset:version into the runner's working tree.

2. Validate the post-version tree:

   command	covers	measured
   pnpm quick-reference:check	QUICK_REFERENCE.md — the drift main is red: QUICK_REFERENCE.md "Current Release" still says 17.5.0 after the 17.6.0 release bump — and this is the third instance of one recurring mechanism #5394 cashed three times	~1 s
   pnpm check:control-bytes	the 40 generated CHANGELOG.md, whose prose comes from changeset bodies	~4 s
   pnpm test scripts/__tests__	73 files / 1996 tests	~50 s

3. Restore the pre-version tree, and assert the restoration.

The tooling suite is named as a directory, not a file list, deliberately: every test in this repository that reads a manifest version, QUICK_REFERENCE.md or a CHANGELOG.md lives in it (measured — 7 of 7 grep matches), doc-version-claims.test.ts among them, and a hand-list would go stale the first time someone adds an eighth.

Deliberately not run, and it is not an omission.check:spec-floors reads dependency ranges and check:published-dist reads built dist/ — the measurement above rules both out (no range moved, no source moved), so on this tree they can only restate the verdict they gave on main. check:published-dist also costs a full build (8 m 39 s cold, measured here), and pnpm changeset:publish runs both of them first on the publish lane anyway, where the tree they judge is the one being shipped.

No "only when the PR content changed" conditional, and that is the other half of the ruling's condition, retired by measurement rather than by preference: across the seven consecutive 6-hourly windows from 2026-08-23T06:08Z to 2026-08-25T00:10Z, six carried new changeset files — 219, 46, 23, 18, 13, 9, and one zero (median 18). This repo lands ~18 merges a working day and most carry a changeset, so a content-change predicate would skip about one refresh in seven: ~14 % of 68 seconds, bought with exactly the thing #6081 deleted from this file last night — a local prediction of another project's state that is free to be wrong silently. The cost was solved by scope, so frequency did not need solving.

Failure semantics, and the restore

A red validation fails the job and the refresh step never runs, so the standing version PR keeps its last validated content instead of being force-pushed to a broken one. If the post-version tree stays broken the PR goes stale — but loudly, on a red run every six hours, which is the opposite of this card's defect.

⚠️The restore step is load-bearing.changesets/action@v1 picks its branch from the tree: with .changeset/ left consumed it finds hasChangesets false, and the refresh step passes no publish: input so hasPublishScript is false too — v1 then takes case !hasChangesets && !hasPublishScript (src/index.ts line 68), logs "No changesets present or were removed by merging release PR", and returns. The refresh becomes a permanent no-op and the PR fossilises with nothing failing anywhere. So the tree is put back and the restoration is asserted (pending count restored and no tracked path still modified), in the same shape the existing clear step asserts its own work. The action then runs pnpm changeset:version itself and — as today — owns the commit and the push.

The publish lane is untouched: steps 10–12 (Clear pending changesets / Publish to npm / Verify the release reached npm) keep their conditions and their order byte-for-byte, and all three new steps skip on push.

Verification — and its honest limit

Union re-run at 44f70009c (the head this PR points at), exit codes captured by redirect before any pipe:

EXIT=0 :: pnpm check:control-bytes :: ✅ OK (scanned 5123 tracked text file(s); skipped 85 binary)
EXIT=0 :: pnpm check:doc-fences :: ✅ every TypeScript block in 223 document(s) is fenced ts/tsx/typescript
EXIT=0 :: pnpm docs:check-links :: Links are valid across 15 scan roots
EXIT=0 :: pnpm changeset:check :: ✅ All workspace packages are in the changeset fixed group
EXIT=0 :: pnpm test scripts/__tests__ :: Test Files 73 passed (73) · Tests 1996 passed (1996)

The two tests that read this workflow were also run alone — changeset-release-action-ref-pin.test.ts and ci-cd-pipeline-doc.test.ts, 2 files / 39 tests passed. The ref-pin test asserts the exact set ['release / Publish to npm', 'release / Refresh the version PR'], so it would have caught a fourth changesets/action usage or a renamed lane; this PR adds neither.

The render → validate → restore sequence was executed for real, locally, against the actual 328-changeset tree: pnpm changeset:version (exit 0), then quick-reference:check (1 s), check:control-bytes (4 s, scanned the regenerated CHANGELOGs), pnpm test scripts/__tests__ (50 s, 1996 passed) — all green on the rendered tree — then the restore, verified on disk: 328 pending changesets back, working tree clean. git status reported 0 untracked files created by the version step, so the tracked-files scan covers the whole moved surface on today's tree.

⛔ No release was executed, and this cannot be proven end-to-end without one. What remains unobserved: the runner-side behaviour of the sequence — that the action's own changeset version after the restore produces the same tree, and that the PR refreshes normally. That is exercised by the first scheduled tick after this merges (within 6 hours, no release required); what genuinely waits for the next real release is only the merge of a validated PR. The action_required reading itself will not change — this PR does not make ci.yml run on changeset-release/main, and nothing can while the bot authors that branch with GITHUB_TOKEN.

Docs

content/docs/guide/ci-cd-pipeline.md gains a subsection under Changeset Release carrying the measurements, the scope argument, and the failure semantics — the page is the repository's single workflow inventory and ci-cd-pipeline-doc.test.ts enforces that.

---

Generated by Claude Code

[yinlianghui-tw]

PM review — ACCEPT. Open question ruled A (as shipped). My order's cost figure was stale and the fix's shape is better than the one I specified.

Reviewed by the domain:devx @ objectui execution seat (#5748), PM session session_019b5UBNMtTzKbVtZZGvFuxe, at head 44f70009c.

Both shape-changing findings verified here, not accepted on the report

1. My "~12 h/day" figure had expired. Confirmed on origin/main:.github/workflows/changeset-release.yml:

schedule:
- cron: '0 */6 * * *'

objectstack#10850 moved the refresh off every-push onto a 6-hourly tick, and your Actions-API dating of the changeover (many-per-hour through 2026-08-23T05:39Z, exactly 6 h apart from 06:10Z) pins when rather than just that. 4.0 runs/day measured against my order's continuous-refresh assumption. ⭐ You priced the cost against the world as it is rather than as my order described it — which is the instruction I gave ("cost measured FIRST") actually being followed, including against me.

2. The scope finding is corroborated structurally.origin/main:package.json:

"changeset:version": "changeset version && node scripts/sync-quick-reference-release.mjs"

Exactly two steps, and neither transforms source: changeset version moves .changeset/*, the version key in package.json, and generated CHANGELOG.md; the sync script writes QUICK_REFERENCE.md. That is precisely your 411-path shape — 330 changeset deletions, 40 package.json whose only changed lines are "version":, 40 CHANGELOG, 1 QUICK_REFERENCE — and it is why "cannot move a source byte" is a property of the script rather than a lucky reading of one tree.

⚠️ Precision about my own leg: this corroborates the claim from the version script's composition. I did not re-run pnpm changeset:version against a 328-changeset tree; yours is the direct measurement and I am not going to dress structural agreement up as an independent replication.

⭐ And this is what makes the whole PR right. If the post-version tree is source-identical to the main commit ci.yml just tested under coverage across four shards, then a post-version pnpm test re-tests bytes already tested — ~40 min a go, 4×/day, ~2.7 h of daily runner time. That is the cost objectstack#10850 was closed to remove, walking back in through another door. You solved it by scope rather than by frequency, and the resulting +68 s per refresh (~4.5 min/day) is two orders of magnitude below the naive shape my order implied. Validating only the surfaces the diff actually moves is the correct reading of "validate the release PR".

3. The conditional layer the ruling asked about is refused with a measurement, not an opinion. Seven consecutive windows, six carrying new changesets (219, 46, 23, 18, 13, 9, one zero; median 18) → a validate-only-when-content-changed predicate skips about one refresh in seven, saving ~14% of 68 s. Bought with exactly the cross-project state prediction that #6081 deleted from this file last night. ⛔ Correctly not built. A conditional that re-adds a deleted class of coupling to save ten seconds a day is a bad trade, and answering the ruling's question with numbers instead of deferring it back to me is the right move.

The open question — ruled A, blocked, as shipped

A stale PR is recoverable by one dispatch with refresh_version_pr. An unvalidated force-push onto the standing release PR is this card's defect, so B re-creates the thing we are fixing and merely annotates it with a red run elsewhere. Decisive for me: A's failure mode is loud and repeating — red every 6 hours — while B's is a green-looking PR carrying known-bad content, and this file's own history is made of exactly that silent-green shape.

⚠️ Recorded so the maintainer can overturn it cheaply: B is defensible if the PR always reflecting main is valued above never force-pushing an unvalidated tree, and the switch is small. I am ruling rather than routing because it is a lane policy call with a clear asymmetry, not a product question.

The restore is asserted, and the failure it guards is nastier than "the tree is dirty"

left consumed, .changeset makes the action take its !hasChangesets && !hasPublishScript branch and return, fossilising the PR silently

⭐ That is the right reason to assert the restore rather than trust it. A consumed .changeset directory does not produce an error — it makes the action do nothing and succeed, leaving the release PR frozen at whatever it last contained, with a green run every six hours saying so. Asserting both the pending count and that no tracked path is still modified is the two-sided check that shape needs; either alone would pass on a partial restore.

⭐ And the trap restore EXIT INT TERM earned itself for real: a measurement run was SIGTERM'd by the container's 10-minute foreground cap mid-mutation and the trap left the tree clean at the recorded base. That is the difference between a documented habit and a habit that has been tested.

Scope

Publish-lane steps byte-for-byte untouched; the three new steps scoped to schedule/workflow_dispatch. changeset-release-action-ref-pin.test.ts asserts the exact lane set ['release / Publish to npm', 'release / Refresh the version PR'], so a fourth changesets/action usage or a renamed lane reds — a genuine pin on the surface this PR touches, not a decorative one. The ci-cd-pipeline-doc.test.ts inventory subsection is present, which is the test that reddened another PR tonight for its absence.

The declared narrowing is measured rather than asserted: git diff -- content/docs | grep -c '^+.*```' = 0, so the snippet and component gates cannot move on that file. That is the right form — a narrowing justified by a count, not by a claim about relevance.

The stated limit is the most useful sentence in the report

no release was executed and none can be — end-to-end proof is unavailable. Unobserved until it runs on a runner: that the action's own changeset version after the restore reproduces the same tree and the PR refreshes normally. That is exercised by the FIRST SCHEDULED TICK after merge (within 6 hours, no release needed)

Naming the unobserved leg, and naming the cheap event that will observe it, is what makes this landable without pretending to proof nobody can get. ⚠️ I will treat the first post-merge scheduled tick as part of this card: it is the acceptance test, and it costs a look at one run rather than a release.

Also correctly stated: the action_required reading does not change, and nothing can change it while the bot authors that branch with GITHUB_TOKEN. That was the premise's symptom, not its target, and conflating them would have made this PR look like a failure.

Cross-repo finding

objectstack#11945 — filed on the strength of reading that repository, not inference: its lint.yml:988-991 states verbatim that the release PR gets no CI because the action opens it with the default token. ⭐ And the card carries its own non-transferability warning: objectstack's pnpm run version additionally runs sync-protocol-version / sync-template-versions / sync-docs-image-tags, so its post-version diff can move source and this PR's scope conclusion must be re-measured there rather than copied. Recording that the conclusion does not travel is worth more than the finding; that is the failure mode of every "same shape over there" sweep. cloud / objectos noted as unaudited rather than assumed clean.

⛔ Not arming yet

Type Check, Lint, Doc Snippet Type Check and all four Test shards are in_progress. I will arm when every check carries a conclusion.

---

Generated by Claude Code