Uh oh!
There was an error while loading. Please reload this page.
ci(release): drop the duplicate test run and stop concurrency discarding release runs - #5443
Merged
os-zhuang merged 1 commit intoAug 21, 2026
Conversation
…ing release runs `changeset-release.yml` did two things it should not. It re-ran `pnpm test` against a commit already on `main`. That could never keep anything out — only stop the release afterwards, which it did on 2 of the last 27 release-PR merges (runs #3606 and #3901 failed there and skipped the changesets step). It was 31m19s of a 33m22s job; everything else combined is ~90s. The push lane in `ci.yml` tests the same commit across four coverage shards and enforces the thresholds on the merged report, so this copy was duplication ~34 minutes later. Its concurrency group was shared by every push to `main`, which does not queue: GitHub holds one pending run per group and cancels the rest. Over runs #3712-#3911, 93 of 200 were `cancelled` with an empty jobs array — no step ever ran — on the workflow whose last step publishes to npm. The group is now keyed by commit, so nothing is discarded, and ordering moves into a fail-open wait step that holds a run until every older release run has finished. `cancel-in-progress: true` is deliberately not the fix: it keeps one pending slot by killing a run that may be mid-`changeset publish`. Fixes#5404 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RV6yuVCxymHYE16PL9vQkE
os-zhuang
marked this pull request as ready for review
August 21, 2026 01:14
Uh oh!
There was an error while loading. Please reload this page.
os-zhuang
deleted the
claude/issue-5404-release-lane-tests-and-concurrency
branch
August 21, 2026 01:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#5404
Two changes to
.github/workflows/changeset-release.yml, and nothing else.ci.ymlis untouched,changeset:versionandchangeset:publishare untouched, and no release was executed at any point.Half 1 — the duplicate
pnpm testis goneMeasured from each job's own
started_at/completed_at, over runs #3712-#3911 (200 runs, 2026-08-17T05:35:42Z .. 2026-08-20T18:14:30Z). Of those, 106 executed; medians across them:Run testschangesets/action(version or publish)It re-tested a commit already on
main, so failing it could never keep anything out — it could only stop the release after the fact, and it did. Of the last 27 release-PR merges, 2 failed here and skipped the changesets step entirely:5bf5d2cb6, 2026-08-14) — failedRun testsat 1673s. That commit was the publish-eligible one:.changeset/held no changesets.@object-ui/*@17.5.0reached npm at2026-08-14T15:39:55Z, 6h50m after the run failed, on a later push that happened to still find.changeset/empty.form-view.ts's spec bridge still readsspec.aria, a keyFormViewSchemaretired — a dormant read that can never fire #3901 (59f61cfb8, 2026-08-20) — failedRun testsat 1965s.The backstop is real and was re-confirmed on the commit this branch is based on (
15236e055, ci.yml run 32402092139), not taken on the card's word:All four shards green.
Test (coverage)is red for a reason downstream of enforcement, and its step list says which half:Merge the shard reports into one coverage reportsucceeded — that is where the thresholds are enforced (#5403) — and onlyUpload coverage to Codecovplus the loud outcome step failed. That is the unsetCODECOV_TOKEN, tracked separately in #5436; nothing here touches the upload.Half 2 — the concurrency group discarded runs; now it cannot
concurrency: (workflow)-(ref)shares one group across every push tomain, and a shared group does not queue. GitHub holds at most one pending run per group and cancels the previous pending run when a newer one arrives. Independently reproduced on the same 200 runs: 93 wereconclusion: cancelledwith an empty jobs array — the API returns zero jobs, so not one step ever ran. Counter-probe: the same endpoint returns a populated jobs array for all 106 runs that did execute, so the empty array is a reading and not a gap in the data. Median wait for a run that survived (runcreated_atto its job'sstarted_at): 5m16s.Why not
cancel-in-progress: trueThe last step is
changeset publish. Cancelling keeps the single pending slot by killing the run that may be mid-publish, which is worse than any wait. It is not used here.Is a dropped run harmless because the next one subsumes it? For publishes, no — and this is the part worth reading
changesets/action@v1picks its branch from the checked-out tree (src/index.ts):runVersion, no publishSo the two halves of the release lane subsume differently:
mainand only the release PR removes them, so any later commit's tree is a superset of the discarded run's changesets. A discardedrunVersioncosts nothing: the next run rebuilds the release PR from a tree that still contains everything the dropped one would have consumed..changeset/to be empty, a state that exists only right after a release PR merges and that the very next merged changeset destroys. If that one run is discarded, the next run sees changesets and takesrunVersioninstead — the bumped version is never published, and the following release PR bumps past it.That is not a hypothetical.
@object-ui/core's CHANGELOG declares 90 versions; npm has 74.@object-ui/reactand@object-ui/typesare missing exactly the same 16, so these are whole releases, not per-package failures. Filed separately as #5442, because its dominant cause is a different one this PR is forbidden to touch (a release-merge commit that lands with changesets pending versions again instead of publishing) — but it is the evidence that a discarded publish run is a permanently skipped npm version rather than a delay.The shape
GitHub offers exactly two behaviours for a contended group — cancel the running one, or discard the pending ones — so a queue that actually queues cannot be expressed in the
concurrency:key. This PR splits the two properties the old key was conflating:github.sha, so it holds exactly one run and there is never a pending run to cancel. This is the guarantee, and it is three lines of YAML with no code.Wait for older release runs to finishstep that holds a run until every release run with a lowerrun_numberhas finished.run_numberis a total order, so the oldest run is always free to proceed and this cannot deadlock; FIFO also means the newest run force-pushes the release branch last, so the release PR ends up reflecting the newest tree.The wait step is deliberately fail-open, which is what makes it safe to add to a lane I cannot exercise: every exit path is
exit 0and it also carriescontinue-on-error: true. If the API cannot be read, or an older run is wedged past the 900s cap, it warns and lets the release proceed. Its worst case is today's unserialised behaviour; it can delay a release, never block one. Serialisation is best-effort on top of a guarantee, not the guarantee itself.Half 1 also makes half 2 far cheaper — a ~90s job contends where a 33m22s job could not help but contend — but it does not make it unnecessary, which is why both are here.
Verification
Gate union re-run on the final commit
b808ce85fwith a clean tree:scripts/__tests__/is the whole surface that reads.github/workflows/**and.changeset/**, derived from a three-dot diff against the merge base15236e055(two files changed).Reverse verification of the pin that guards this file.
sync-quick-reference-release.test.tsasserts theversion:input, so a green run has to be shown to be a real reading. From the committed state,version: pnpm changeset:versionwas changed toversion: pnpm changeset version; the expected direction is red, and that is what happened:Restored with
git checkout HEAD -- ...(clean tree afterwards) and green again at 12 passed.The wait loop was exercised, not just written. It cannot run in CI here, so it was run locally against recorded
/actions/workflows/changeset-release.yml/runspayloads with a stand-ingh, six cases:gh apifailsScope
.github/workflows/changeset-release.ymland one changeset file are touched.changeset:versionis unchanged, so fix(release): carry QUICK_REFERENCE's Current Release block in changeset:version #5396's QUICK_REFERENCE sync and its pin are intact — the pin was re-run in both directions above.changeset:publishbehaviour is unchanged: same script, same inputs, sameenv.changeset-release/mainisaction_required, so the release commit reachesmainunvalidated #5397 and 合并队列声称「已强制」却从未产生过一次 merge_group 构建(repo-wide 0),必需集实测不含 4 个 shard / Type Check / Lint —— #3523 的第 3 步从未落地,而 AGENTS.md §9 已按「队列会替你兜住」反转了 auto-merge 禁令 #4986 remain open and untouched;Test (coverage)is red on every push tomainbecauseCODECOV_TOKENis unset — the four shards are green, the upload is rejected #5436 is a maintainer action and no part of this PR.Generated by Claude Code