Skip to content

ci(release): drop the duplicate test run and stop concurrency discarding release runs - #5443

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5404-release-lane-tests-and-concurrency
Aug 21, 2026
Merged

ci(release): drop the duplicate test run and stop concurrency discarding release runs#5443
os-zhuang merged 1 commit into
mainfrom
claude/issue-5404-release-lane-tests-and-concurrency

Conversation

@claude

@claudeclaudeBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Fixes#5404

Two changes to .github/workflows/changeset-release.yml, and nothing else. ci.yml is untouched, changeset:version and changeset:publish are untouched, and no release was executed at any point.

Half 1 — the duplicate pnpm test is gone

Measured from each job's own started_at/completed_at, over runs #3712-#3911 (200 runs, 2026-08-17T05:35:42Z .. 2026-08-20T18:14:30Z). Of those, 106 executed; medians across them:

stepmedianshare
Run tests31m19s94%
Build packages35s
changesets/action (version or publish)24s
Install dependencies6s
Checkout Repo8s
whole job33m22s

It re-tested a commit already on main, so failing it could never keep anything out — it could only stop the release after the fact, and it did. Of the last 27 release-PR merges, 2 failed here and skipped the changesets step entirely:

The backstop is real and was re-confirmed on the commit this branch is based on (15236e055, ci.yml run 32402092139), not taken on the card's word:

success 609s Test (coverage shard 1/4)
success 669s Test (coverage shard 2/4)
success 648s Test (coverage shard 3/4)
success 509s Test (coverage shard 4/4)
failure 53s Test (coverage)

All four shards green. Test (coverage) is red for a reason downstream of enforcement, and its step list says which half: Merge the shard reports into one coverage reportsucceeded — that is where the thresholds are enforced (#5403) — and only Upload coverage to Codecov plus the loud outcome step failed. That is the unset CODECOV_TOKEN, tracked separately in #5436; nothing here touches the upload.

Half 2 — the concurrency group discarded runs; now it cannot

concurrency: (workflow)-(ref) shares one group across every push to main, and a shared group does not queue. GitHub holds at most one pending run per group and cancels the previous pending run when a newer one arrives. Independently reproduced on the same 200 runs: 93 were conclusion: cancelled with an empty jobs array — the API returns zero jobs, so not one step ever ran. Counter-probe: the same endpoint returns a populated jobs array for all 106 runs that did execute, so the empty array is a reading and not a gap in the data. Median wait for a run that survived (run created_at to its job's started_at): 5m16s.

Why not cancel-in-progress: true

The last step is changeset publish. Cancelling keeps the single pending slot by killing the run that may be mid-publish, which is worse than any wait. It is not used here.

Is a dropped run harmless because the next one subsumes it? For publishes, no — and this is the part worth reading

changesets/action@v1 picks its branch from the checked-out tree (src/index.ts):

  • no changeset files at all + a publish script → publish
  • changesets present, all with empty frontmatter → does nothing
  • changesets present → runVersion, no publish

So the two halves of the release lane subsume differently:

  • The version half does subsume. Changesets are files committed to main and only the release PR removes them, so any later commit's tree is a superset of the discarded run's changesets. A discarded runVersion costs nothing: the next run rebuilds the release PR from a tree that still contains everything the dropped one would have consumed.
  • The publish half does not. Publishing requires .changeset/ to be empty, a state that exists only right after a release PR merges and that the very next merged changeset destroys. If that one run is discarded, the next run sees changesets and takes runVersion instead — the bumped version is never published, and the following release PR bumps past it.

That is not a hypothetical. @object-ui/core's CHANGELOG declares 90 versions; npm has 74. @object-ui/react and @object-ui/types are missing exactly the same 16, so these are whole releases, not per-package failures. Filed separately as #5442, because its dominant cause is a different one this PR is forbidden to touch (a release-merge commit that lands with changesets pending versions again instead of publishing) — but it is the evidence that a discarded publish run is a permanently skipped npm version rather than a delay.

The shape

GitHub offers exactly two behaviours for a contended group — cancel the running one, or discard the pending ones — so a queue that actually queues cannot be expressed in the concurrency: key. This PR splits the two properties the old key was conflating:

  1. Nothing is discarded, structurally: the group is keyed by github.sha, so it holds exactly one run and there is never a pending run to cancel. This is the guarantee, and it is three lines of YAML with no code.
  2. Runs are still serialised, by a Wait for older release runs to finish step that holds a run until every release run with a lower run_number has finished. run_number is a total order, so the oldest run is always free to proceed and this cannot deadlock; FIFO also means the newest run force-pushes the release branch last, so the release PR ends up reflecting the newest tree.

The wait step is deliberately fail-open, which is what makes it safe to add to a lane I cannot exercise: every exit path is exit 0 and it also carries continue-on-error: true. If the API cannot be read, or an older run is wedged past the 900s cap, it warns and lets the release proceed. Its worst case is today's unserialised behaviour; it can delay a release, never block one. Serialisation is best-effort on top of a guarantee, not the guarantee itself.

Half 1 also makes half 2 far cheaper — a ~90s job contends where a 33m22s job could not help but contend — but it does not make it unnecessary, which is why both are here.

Verification

Gate union re-run on the final commit b808ce85f with a clean tree:

scripts/__tests__ (58 files) 58 passed | 1571 tests passed
pnpm check:control-bytes OK (4495 tracked text files)
pnpm changeset:check fixed group OK | privatePackages declared | no major
pnpm quick-reference:check Current Release block already states every anchor
check-changeset-presence.mjs no source of a released package changed; nothing owed
check-lint-coverage.mjs 46/46 packages linted

scripts/__tests__/ is the whole surface that reads .github/workflows/** and .changeset/**, derived from a three-dot diff against the merge base 15236e055 (two files changed).

Reverse verification of the pin that guards this file.sync-quick-reference-release.test.ts asserts the version: input, so a green run has to be shown to be a real reading. From the committed state, version: pnpm changeset:version was changed to version: pnpm changeset version; the expected direction is red, and that is what happened:

FAIL the release path actually runs the sync > `changeset-release.yml` versions through `pnpm changeset:version`
AssertionError: ... expected 'pnpm changeset version' to be 'pnpm changeset:version'
Tests 1 failed | 11 passed (12)

Restored with git checkout HEAD -- ... (clean tree afterwards) and green again at 12 passed.

The wait loop was exercised, not just written. It cannot run in CI here, so it was run locally against recorded /actions/workflows/changeset-release.yml/runs payloads with a stand-in gh, six cases:

caseresult
no older run in flightproceeds at once, exit 0
only newer runs in flightproceeds at once — a newer run must not block us
two older runs in flight, cap reachedpolls, warns, proceeds, exit 0
older run finishes mid-waitproceeds as soon as it completes (6s), not at the 120s cap
gh api failswarns, proceeds, exit 0 — fail-open
this run is the oldestproceeds at once

Scope


Generated by Claude Code

…ing release runs
`changeset-release.yml` did two things it should not.
It re-ran `pnpm test` against a commit already on `main`. That could never keep
anything out — only stop the release afterwards, which it did on 2 of the last 27
release-PR merges (runs #3606 and #3901 failed there and skipped the changesets
step). It was 31m19s of a 33m22s job; everything else combined is ~90s. The push
lane in `ci.yml` tests the same commit across four coverage shards and enforces the
thresholds on the merged report, so this copy was duplication ~34 minutes later.
Its concurrency group was shared by every push to `main`, which does not queue:
GitHub holds one pending run per group and cancels the rest. Over runs
#3712-#3911, 93 of 200 were `cancelled` with an empty jobs array — no step ever
ran — on the workflow whose last step publishes to npm. The group is now keyed by
commit, so nothing is discarded, and ordering moves into a fail-open wait step that
holds a run until every older release run has finished.
`cancel-in-progress: true` is deliberately not the fix: it keeps one pending slot
by killing a run that may be mid-`changeset publish`.
Fixes#5404
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RV6yuVCxymHYE16PL9vQkE
@os-zhuang
os-zhuang marked this pull request as ready for review August 21, 2026 01:14
@os-zhuang
os-zhuang added this pull request to the merge queueAug 21, 2026
Merged via the queue into main with commit 478ec54Aug 21, 2026
22 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5404-release-lane-tests-and-concurrency branch August 21, 2026 01:15
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

changeset-release.yml: drop the duplicate pnpm test, and stop concurrency discarding release runs (95 of 200 never executed)

2 participants

@os-zhuang@claude