From 44f70009c195f5801cf30378188bde8b7499bf99 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 25 Aug 2026 02:03:26 +0000 Subject: [PATCH] ci(release): validate the post-version tree before the version PR is refreshed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The release PR runs no CI and cannot be given any. Re-measured 2026-08-25: `ci.yml` has 849 runs on `changeset-release/main`, every recent one `action_required` with `created_at == run_started_at == updated_at` — created and immediately parked, nothing executed. GitHub does not start workflow runs from events raised by `GITHUB_TOKEN`, and the refresh force-pushes that branch, so there is no stable head to re-run against either. On the 17.6.0 release PR the check-runs endpoint returned `total_count: 1`: one job, 7 seconds after the merge. The release commit is the only commit reaching `main` without passing the merge queue. Maintainer ruling 2026-08-22 (option A): validate the post-version tree inside this workflow, before `changesets/action` opens or updates the PR. The ruling attached a pre-condition — price it first — and the price chose the shape. The version step cannot move a source byte. Rendered against the real tree (328 pending changesets, 17.6.0 -> 17.7.0) it touches 411 paths: 330 `.changeset/*.md` deleted, 40 `package.json` (the `"version"` key and nothing else — 80 changed lines, all of them `"version":`), 40 generated `CHANGELOG.md`, and `QUICK_REFERENCE.md`. The source is byte-identical to the `main` commit whose push lane `ci.yml` just ran under coverage across four shards, so a `pnpm test` here would re-test tested bytes at ~40 min a go (ci.yml's own instrumented figure: 39m51s of tests), four times a day on the 6-hourly cron — ~2.7 h of daily runner time for a PR nobody reads until release day, i.e. the cost objectstack#10850 was closed to remove, walking back in. So the refresh lane renders the version itself, validates the surfaces the diff can move, and restores the tree for the action: pnpm quick-reference:check QUICK_REFERENCE.md ~1 s pnpm check:control-bytes the 40 generated CHANGELOGs ~4 s pnpm test scripts/__tests__ 73 files / 1996 tests ~50 s The suite is named as a directory, not a file list: every test that reads a manifest version, `QUICK_REFERENCE.md` or a `CHANGELOG.md` lives in it (7 of 7 matches), `doc-version-claims.test.ts` among them. `check:spec-floors` and `check:published-dist` are deliberately absent — they read dependency ranges and built `dist/`, neither of which moved, and `changeset:publish` runs both first on the publish lane. No "only when the PR content changed" condition, for a measured reason: across the seven consecutive 6-hourly windows from 2026-08-23T06:08Z to 2026-08-25T00:10Z, six carried new changeset files (median 18). Such a predicate would skip about one refresh in seven while adding the local prediction of another project's state that #6081 just deleted from this file. Failure semantics: a red validation fails the job and the refresh never runs, so the PR keeps its last validated content. Nothing can touch the publish lane — all three steps are scoped to `schedule` / `workflow_dispatch`. The restore is load-bearing and asserted, not assumed: with `.changeset/` left consumed the action finds nothing pending, takes its no-op branch and returns, and the PR fossilises with nothing failing anywhere. Part of #5397 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019b5UBNMtTzKbVtZZGvFuxe --- ...5397-release-pr-post-version-validation.md | 8 + .github/workflows/changeset-release.yml | 201 ++++++++++++++++++ content/docs/guide/ci-cd-pipeline.md | 48 +++++ 3 files changed, 257 insertions(+) create mode 100644 .changeset/5397-release-pr-post-version-validation.md diff --git a/.changeset/5397-release-pr-post-version-validation.md b/.changeset/5397-release-pr-post-version-validation.md new file mode 100644 index 0000000000..06dfedf041 --- /dev/null +++ b/.changeset/5397-release-pr-post-version-validation.md @@ -0,0 +1,8 @@ +--- +--- + +CI-only: the changeset release workflow's refresh lane now renders the +post-version tree, validates the surfaces `changeset version` can move +(`QUICK_REFERENCE.md`, the generated `CHANGELOG.md` files, the manifest +versions as the tooling suite reads them) and restores the tree before +`changesets/action` opens or updates the version PR. No package changes. diff --git a/.github/workflows/changeset-release.yml b/.github/workflows/changeset-release.yml index 7e7e10f263..8513a64901 100644 --- a/.github/workflows/changeset-release.yml +++ b/.github/workflows/changeset-release.yml @@ -92,6 +92,17 @@ name: Changeset Release # someone can get wrong. # dispatch without the input -> NOTHING, loudly. See the input's note. # +# THE REFRESH LANE VALIDATES WHAT IT IS ABOUT TO RENDER (objectui#5397) +# --------------------------------------------------------------------- +# The release PR runs no CI and cannot be made to — 849 `ci.yml` runs on +# `changeset-release/main`, every recent one `action_required`, created and +# parked with nothing executed — so the release commit is the only commit that +# reaches `main` without passing the merge queue. Before the refresh step +# invokes the action, this job therefore renders the post-version tree itself, +# validates the surfaces the version step can move, and puts the tree back. The +# ruling, the measurements that chose that scope over a 40-minute `pnpm test`, +# and why the restore is load-bearing are at "POST-VERSION VALIDATION" below. +# # WHY THE PUBLISH LANE IS KEYED ON npm AND NOT ON `.changeset/` # ------------------------------------------------------------- # It used to be keyed on "does this commit carry pending changesets?", and that @@ -844,6 +855,196 @@ jobs: echo "::error::${ANCHOR_PKG}@${EXPECTED} is STILL not on npm after a publish run that reported success. The repository declares a version the registry has never seen — this is objectui#5442's silent failure, made loud. Check the 'Publish to npm' step's log for which branch changesets/action took." exit 1 + # ══════════════════════════════════════════════════════════════════════ + # POST-VERSION VALIDATION — THE ONLY GATE THE RELEASE COMMIT EVER PASSES + # (objectui#5397) + # ══════════════════════════════════════════════════════════════════════ + # The release PR runs NO CI, and cannot be made to. Re-measured + # 2026-08-25: `ci.yml` has 849 runs on `changeset-release/main` and every + # recent one is `action_required` with + # `created_at == run_started_at == updated_at` — created and immediately + # parked, nothing executed. GitHub does not start workflow runs from + # events raised by `GITHUB_TOKEN`, and the refresh step below force-pushes + # that branch, so there is no stable head for a human to re-run against + # either. On the 17.6.0 release PR the check-runs endpoint returned + # `total_count: 1` — one job, started 7 SECONDS AFTER the merge. The + # largest machine-generated diff of the week is therefore the only commit + # that reaches `main` without passing the merge queue. + # + # Maintainer ruling 2026-08-22 (「接受所有」): validate the POST-VERSION + # tree HERE, before the action opens or updates the PR. Not by giving the + # PR a PAT / GitHub App token (widens the supply-chain trust surface and + # adds rotation obligations), and not by guaranteeing outputs one artifact + # at a time (objectui#5396 saved exactly one doc row and did not + # generalise). The ruling attached a condition — PRICE IT FIRST — and the + # price is what chose the shape below. + # + # WHY THIS IS NOT `pnpm test` (measured 2026-08-25, not argued) + # ------------------------------------------------------------ + # The naive reading of the ruling is "run the suite on the post-version + # tree". It is the wrong shape, because THE VERSION STEP CANNOT MOVE A + # SOURCE BYTE. Rendered against this repository's real tree (328 pending + # changesets, 17.6.0 -> 17.7.0) `pnpm changeset:version` touches 411 paths + # and every one of them is one of four kinds: + # + # 330 .changeset/*.md deleted (consumed by the version) + # 40 */package.json the `"version"` key and NOTHING else — + # 80 changed lines, all of them `"version":` + # 40 */CHANGELOG.md generated from the changeset bodies + # 1 QUICK_REFERENCE.md rewritten by `changeset:version`'s + # `sync-quick-reference-release.mjs` + # + # Zero `.ts`, zero `.tsx`, zero config. The source in the post-version + # tree is byte-identical to the `main` commit this run checked out, and + # `ci.yml`'s push lane already ran the whole suite over that commit under + # coverage across four shards. A `pnpm test` here would re-test bytes that + # were tested ~13 min ago and would report nothing new — at 40 minutes a + # go (`ci.yml` measured its own unsharded suite at 39 min 51 s of tests / + # 40 min 19 s of job), four times a day on the cron below, i.e. ~2.7 h of + # runner time daily for a PR nobody reads until release day. That is the + # cost objectstack#10850 was closed to REMOVE, walking back in. + # + # So the validation is scoped to what the diff can actually move, and the + # union — `ci.yml` on the source, these steps on the version outputs — is + # the whole post-version tree. + # + # pnpm quick-reference:check QUICK_REFERENCE.md ~1 s + # pnpm check:control-bytes the 40 generated CHANGELOGs ~4 s + # pnpm test scripts/__tests__ 73 files / 1996 tests ~50 s + # + # The tooling suite is named as a DIRECTORY rather than as a list of test + # files, deliberately: every test in this repository that reads a + # manifest version, `QUICK_REFERENCE.md` or a `CHANGELOG.md` lives in it + # (measured — 7 of 7 matches), `doc-version-claims.test.ts` among them, + # and a list would go stale the first time someone adds an eighth. + # Measured green on a real rendered tree: 1 s + 4 s + 50 s. + # + # ⛔ WHAT IS DELIBERATELY NOT RUN, AND WHY IT IS NOT AN OMISSION. + # `check:spec-floors` and `check:published-dist` read dependency RANGES + # and built `dist/` output. The measurement above is what rules them out: + # no range moved (the only changed key is `"version"`) and no source moved, + # so on this tree they can only restate the verdict they gave on `main`. + # `check:published-dist` also costs a full build (8 m 39 s cold, measured), + # and `pnpm changeset:publish` runs both of them first on the publish lane + # anyway, where the tree they judge is the one being shipped. + # + # ⛔ WHY THERE IS NO "ONLY WHEN THE PR CONTENT CHANGED" CONDITION. That + # layer was the other half of the ruling's condition, and the measurement + # retired it: across the seven consecutive 6-hourly windows from + # 2026-08-23T06:08Z to 2026-08-25T00:10Z, SIX carried new changeset files + # (median 18 per window; 219, 46, 23, 18, 13, 9, and one zero). This + # repository lands ~18 merges a working day and most carry a changeset, so + # a content-change predicate would skip about one refresh in seven — it + # would buy ~14% of 70 seconds while adding exactly the thing + # objectui#6081 just deleted from this file: a local prediction of another + # project's state that is free to be wrong silently. The cost was solved + # by SCOPE, so frequency does not need solving. + # + # FAILURE SEMANTICS. A red validation fails the job and the refresh step + # below never runs, so the standing PR keeps its last VALIDATED content + # instead of being force-pushed to a broken one. If the post-version tree + # stays broken the PR goes stale — but loudly, on a red run every six + # hours, which is the opposite of this card's defect. Nothing here can + # block the publish lane: every step is scoped to `schedule` / + # `workflow_dispatch`, and `push` skips all three. + # + # ⚠️ THE RESTORE STEP IS LOAD-BEARING — DO NOT DROP IT. These steps render + # the version into the runner's working tree, and `changesets/action` + # picks its branch from that tree: with `.changeset/` already consumed it + # would find `hasChangesets` false, and with no `publish:` input on the + # refresh step `hasPublishScript` is false too, so v1 takes + # `case !hasChangesets && !hasPublishScript` (`src/index.ts` line 68), + # logs "No changesets present or were removed by merging release PR", and + # RETURNS. The refresh becomes a permanent no-op and the PR fossilises + # with nothing failing anywhere — this file's own failure class, one lane + # over. So the tree is put back and the restoration is ASSERTED rather + # than assumed, in the same shape the clear step asserts its own work. + # The action then runs `pnpm changeset:version` itself, on the inputs + # these steps measured, and it — not this job — owns the commit and the + # push. + - name: Render the post-version tree + id: post_version + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + run: | + set -euo pipefail + shopt -s nullglob + + # `/^README\.md$/i` is how `@changesets/read` spells its exclusion, so + # the comparison is case-folded rather than matching one spelling. + count_pending() { + local n=0 file + for file in .changeset/*.md .changeset/pre/*.md; do + if [ "$(printf '%s' "${file##*/}" | tr '[:upper:]' '[:lower:]')" = 'readme.md' ]; then + continue + fi + n=$((n + 1)) + done + printf '%s' "$n" + } + + pending_before=$(count_pending) + echo "pending_before=${pending_before}" >> "$GITHUB_OUTPUT" + + pnpm changeset:version + + changed=$(git status --porcelain | wc -l | tr -d ' ') + { + echo "- post-version tree rendered from \`${GITHUB_SHA}\`" + echo "- pending changesets consumed: \`${pending_before}\`" + echo "- paths the version step moved: \`${changed}\`" + } >> "$GITHUB_STEP_SUMMARY" + + - name: Validate the post-version tree + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + run: | + set -euo pipefail + pnpm quick-reference:check + pnpm check:control-bytes + pnpm test scripts/__tests__ + + - name: Restore the pre-version tree + if: always() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') + env: + PENDING_BEFORE: ${{ steps.post_version.outputs.pending_before }} + run: | + set -euo pipefail + shopt -s nullglob + + if [ -z "${PENDING_BEFORE}" ]; then + echo "The render step did not run, so there is nothing to restore." + exit 0 + fi + + git checkout -- . + # Narrow on purpose: the version step can only create files beside a + # package manifest, and a `git clean` in a job that also publishes + # should never be able to reach further than the thing it undoes. + git clean -fdq -- packages apps examples .changeset + + count_pending() { + local n=0 file + for file in .changeset/*.md .changeset/pre/*.md; do + if [ "$(printf '%s' "${file##*/}" | tr '[:upper:]' '[:lower:]')" = 'readme.md' ]; then + continue + fi + n=$((n + 1)) + done + printf '%s' "$n" + } + + pending_after=$(count_pending) + # `--untracked-files=no`: the build that ran earlier in this job + # leaves its own untracked output around, and none of it is part of + # what the version step moved or of what the action reads. Tracked + # state plus the pending count is the whole restoration. + dirty=$(git status --porcelain --untracked-files=no | wc -l | tr -d ' ') + + if [ "${pending_after}" != "${PENDING_BEFORE}" ] || [ "${dirty}" -ne 0 ]; then + echo "::error::The pre-version tree was not restored (${PENDING_BEFORE} changesets before, ${pending_after} after; ${dirty} tracked path(s) still modified). changesets/action would read this tree, find nothing pending and return without refreshing the PR — a silent permanent no-op (objectui#5397)." + exit 1 + fi + echo "Pre-version tree restored: ${pending_after} pending changeset(s), working tree clean." + # REFRESH. No `publish:` input, so `runPublish` is unreachable; no npm # credentials in `env:`, so it is unreachable a second time over. Both are # deliberate — this lane runs unattended on a clock, and the release act in diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index 6e70bb68ea..7f89714013 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -936,6 +936,54 @@ the Published Dist Gate above. A published package whose `dist/` carries tooling the publish before a single tarball reaches npm, which is where that defect actually costs anything ([#4846](https://github.com/objectstack-ai/objectui/issues/4846)). +#### The release PR runs no CI, so the refresh lane validates the tree itself + +The version PR gets **no checks of its own, and cannot be given any**. Measured 2026-08-25: +`ci.yml` has **849** runs on `changeset-release/main` and every recent one is `action_required` +with `created_at == run_started_at == updated_at` — created and immediately parked, nothing +executed. GitHub does not start workflow runs from events raised by `GITHUB_TOKEN`, and the +refresh force-pushes that branch, so there is no stable head to re-run against either. On the +17.6.0 release PR the check-runs endpoint returned `total_count: 1`: one job, started **7 seconds +after the merge**. The release commit is therefore the only commit that reaches `main` without +passing the merge queue ([#5397](https://github.com/objectstack-ai/objectui/issues/5397)). + +So the refresh lane renders `pnpm changeset:version` into the runner's working tree, validates it, +restores the tree, and only then invokes the action — which does its own versioning and owns the +commit and the push. + +**What it validates, and why that is not `pnpm test`.** The version step cannot move a source +byte. Measured against the real tree (328 pending changesets, 17.6.0 → 17.7.0) it touches 411 +paths: 330 `.changeset/*.md` deleted, 40 `package.json` (the `"version"` key and nothing else), 40 +generated `CHANGELOG.md`, and `QUICK_REFERENCE.md`. The source in the post-version tree is +byte-identical to the `main` commit `ci.yml`'s push lane just tested under coverage across four +shards, so a suite run here would re-test tested bytes at ~40 minutes a go, four times a day — +~2.7 h of daily runner time for a PR nobody reads until release day, which is the cost +[objectstack#10850](https://github.com/objectstack-ai/objectstack/issues/10850) was closed to +remove. The validation is scoped to the surfaces the diff can move instead: + +| Command | Covers | Measured | +|---|---|---| +| `pnpm quick-reference:check` | `QUICK_REFERENCE.md` | ~1 s | +| `pnpm check:control-bytes` | the 40 generated `CHANGELOG.md` | ~4 s | +| `pnpm test scripts/__tests__` | 73 files / 1996 tests — every test that reads a manifest version, `QUICK_REFERENCE.md` or a `CHANGELOG.md` | ~50 s | + +`check:spec-floors` and `check:published-dist` are deliberately **not** here: they read dependency +ranges and built `dist/`, neither of which the version step moves, and `pnpm changeset:publish` +runs both first on the publish lane anyway. + +There is also **no "only when the PR content changed" condition**, for a measured reason: across +the seven consecutive 6-hourly windows from 2026-08-23T06:08Z to 2026-08-25T00:10Z, six carried +new changeset files (median 18). Such a predicate would skip about one refresh in seven while +adding a local prediction of another project's state that can be wrong silently — the shape +[#6081](https://github.com/objectstack-ai/objectui/issues/6081) just deleted from this file. + +**Failure semantics.** A red validation fails the job and the refresh never runs, so the standing +PR keeps its last validated content rather than being force-pushed to a broken one. Nothing here +can touch the publish lane: all three steps are scoped to `schedule` / `workflow_dispatch`. The +restore step is load-bearing — with `.changeset/` left consumed the action would find nothing +pending, take its no-op branch and return, and the PR would fossilise with nothing failing +anywhere — so the restoration is asserted, not assumed. + Both lanes configure a pnpm-lock.yaml merge driver to prevent lock file conflicts. ### Published Dist Gate (`published-dist-gate.yml`)