Skip to content

ci: materialise pnpm from a cached Corepack store instead of downloading per job - #11370

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-11300-corepack-queue-ejection
Aug 23, 2026
Merged

ci: materialise pnpm from a cached Corepack store instead of downloading per job#11370
os-steve merged 1 commit into
mainfrom
claude/issue-11300-corepack-queue-ejection

Conversation

@claude

@claudeclaudeBot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Fixes#11300

Measurement first

The card asked how many recent queue failures actually carry this signature. Measured over
merge_group runs from 2026-08-21T00:45Z to 2026-08-23T09:02Z, classified at step level
via the jobs API (not by log grep, so nothing is missed for want of a parseable test name):

failed merge_group runs in window210
failed jobs in those runs230
jobs that died materialising pnpm5, across 4 runs
queue builds in window~450 (1350 runs / 3 workflows)
Corepack downloads in window~10,800 (450 builds x 24 jobs)

Per-job crash rate 0.046%; across 24 draws that is 1.11% per queue build, predicting
5.0 ejection-causing builds over the window against 4 observed. The premise holds — this
is a recurring ~2/day tax, not a one-off.

Verify pnpm version is a sound proxy: it runs only pnpm --version, so the only way it fails
is Corepack materialisation.

The card's signature undercounts the defect by one failure mode. Four of the five carry the
undici assertion; the fifth is the same Corepack download failing differently:

Error when performing the request to https://registry.npmjs.org/pnpm/-/pnpm-10.31.0.tgz
[cause]: TypeError: fetch failed
[cause]: Error: Client network socket disconnected before secure TLS connection
was established { code: 'ECONNRESET', host: 'registry.npmjs.org' }

So the right frame is "Corepack downloads pnpm per job", not "undici has a bug" — the
assertion is one of at least two ways that download dies.

Amplification worth naming: each primary crash also reds the aggregator gates downstream, which
fail on the missing shard attestation (test-4-of-6 MISSING) rather than on anything real.
One network hiccup manufactures several red checks.

File surface — the card undercounted it

The card named three Verify pnpm version sites. The tree has six, plus seven
corepack enable, the seventh (temporal-conformance) having no paired verify step. All seven
are changed; a three-site patch would have left four live and made the survivors look deliberate.

The fix

.github/actions/setup-pnpm restores the Corepack store (COREPACK_HOME) from the actions
cache, keyed on the packageManager pin, then materialises pnpm from it. A warm store never
contacts the registry.

Why this and not the alternatives.

  • Corepack-store cache (chosen). Removes the network call itself on the happy path. Cache
    restore is a different failure domain (GitHub's own cache service) and is non-fatal — a
    restore failure degrades to a download rather than failing the job.
  • setup-nodepackageManager-aware caching (rejected). That caches the pnpm store
    (dependencies), not the pnpm binary; this repo already does it in the adjacent
    Setup pnpm cache steps, and it did not prevent any of the five failures.
  • Retry alone (rejected as the primary). Keeps 24 registry calls per build — it treats the
    symptom and leaves the exposure intact. It is kept strictly as the cold-cache backstop,
    which matters only on the first build after a packageManager bump, when every job misses at
    once.
  • Pinning a different Node (rejected). CI is pinned to Node major 22 via .nvmrc and
    check:node-version; the crash is in that major's bundled undici, and the ECONNRESET variant
    shows the download is fragile regardless of which crash mode Node happens to produce.

actions/setup-node deliberately stays in the workflow at all 7 sites.
scripts/check-node-version.mjs scans .github/workflows/*.ymlonly, and reports how many
setup-node steps it audited. Folding those steps into the composite would have dropped 7 from its
census while it still printed OK — a gate silently auditing less than it claims. Its reading is
unchanged: 32 setup-node steps across 26 workflows, before and after.

What the local evidence covers, and what only the queue can prove

Proven locally (mechanism, with a live control):

legconditionresult
controlcold store + unreachable registryexit 1 in installVersion/fetchTarballURLAndSignature — the same code path as the CI crash
fixwarm store + unreachable registrycorepack install and pnpm --version both exit 0, 10.31.0
retrycold + unreachable3 attempts, 16s (5s+10s backoff), correct ::warning::/::error:: annotations, exit 1 — a permanent failure is not masked

The composite's two shell bodies were extracted from the shipped action.yml and executed
against a simulated Actions environment (GITHUB_OUTPUT/GITHUB_ENV/RUNNER_TEMP), so the
tested text is the shipped text.

Only the queue can prove: that actions/cache restores this path across real runners, the
real cache hit rate under 24-way concurrency, and the first cold-key build (24 simultaneous
misses, each saving ~22MB — expected once per packageManager bump).

Gates

node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack derived 10 families from
the real changeset. All 10 green, plus check:nul-bytes, re-run at final commit 5742da3:

check-node-version: OK (32 setup-node step(s) across 26 workflow(s), all on Node 22).
check-required-contexts: 6 required context name(s) pinned across 2 workflow(s)
check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s)
check-workflow-status-functions: OK (scanned 26 workflow file(s), 49 job(s), ...)
check-aggregator-roster: 3 aggregator(s) across 2 workflow(s); roster == needs: in both directions
check-ci-filter-parity: OK: all 88 declared cross-package glob(s) ...
check-step-collectors: 345 `run:` steps across 26 workflow(s)
check-pnpm-filter-targets: 120/148 `--filter` occurrence(s) across 25 file(s) resolve
check-nul-bytes: OK (scanned 6381 text file(s) ... no raw ASCII control bytes)

Repo-wide pnpm lint is provably irrelevant to this diff rather than skipped: eslint's own
config reports File ignored because no matching configuration was supplied for both changed
paths, so neither is in the linted population, and since the diff changes only .yml no verdict
on any untouched file can move.

Scope

lint.yml (5 sites) and spec-liveness-check.yml (1 site) carry the same defect — together the
other 7 of the 24 jobs in a queue build — but are outside this card's declared file surface.
Filed separately as #11369 so the composite can be adopted there without widening this PR.


Generated by Claude Code

…ing per job
`corepack enable` only writes shims; the pnpm tarball is fetched from
registry.npmjs.org on the first pnpm invocation in the job. A merge-queue build
is 24 jobs, so one merge attempt made 24 independent registry calls before any
test ran, and any one of them could eject the PR and force every PR behind it
to rebuild.
Add a `.github/actions/setup-pnpm` composite action that restores the Corepack
store (COREPACK_HOME) from the actions cache, keyed on the packageManager pin,
then materialises pnpm from it. A warm store makes the happy path fully
network-free. A bounded retry covers the cold-cache case only.
Applied uniformly to all 7 Corepack sites in ci.yml, including the one at the
temporal-conformance job that had no paired verify step.
`actions/setup-node` deliberately stays in the workflow at all 7 sites:
scripts/check-node-version.mjs scans .github/workflows/*.yml only and reports
how many setup-node steps it audited, so moving those steps into the composite
would drop them from its census while it still printed OK.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 23, 2026
@os-steve
os-steve marked this pull request as ready for review August 23, 2026 14:53
@os-steve
os-steve added this pull request to the merge queueAug 23, 2026
Merged via the queue into main with commit a872ce9Aug 23, 2026
43 checks passed
@os-steve
os-steve deleted the claude/issue-11300-corepack-queue-ejection branch August 23, 2026 15:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cdsize/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

1 participant

@os-steve