You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[finding] Merge-queue check-set parity for tree-global ratchets — measure which lint.yml jobs actually run on merge_group, then close the gap the stale-ledger outage rode through #12211
Filed by the spec PM seat (session_01NDGG54XF5gbTLdQzCtnaVV) after the 2026-08-25 stale-ledger outage (~70 min of fleet-wide red; incident cards: the p0 card, its two sibling filings, fix PRs #12186/#12192). Dedup: the incident cards asked triage to consider "the class fix" but no dedicated card exists; #4897 (closed) is the historical precedent that first wired merge_group into workflows.
⚠️ STATUS (PM seat #6023, 2026-08-25).Step 1 is done and it is a NEGATIVE result: step 2's premise is false. PR #12338 (9fffd327c7, Part of — deliberately not Fixes) landed the measurement. No tree-global ratchet is missing merge_group coverage, and adding a trigger anywhere would not have prevented the outage. Exactly four workflows carry merge_group as an on: trigger — ci.yml, governed-surface-guard.yml, lint.yml, spec-liveness-check.yml — all six required contexts live in ci.yml + lint.yml, and no gate job or step inside them is event-gated. The two contradictory observations are resolved below. The residual is two repository SETTINGS facts, both maintainer-only, now in the decision box as #12343. This card is blocked on that ruling, not on any further measurement.
The mechanism this card closes
The gate that went red (check:exported-any-returns) is a tree-global ratchet: it judges the whole tree's consistency (ledger vs built dist), not the diff. For that gate class, PR-level green is a snapshot — a PR branched before a new gate lands (or before main's state changes) carries stale green. The only place that sees the REAL merge state is the merge queue. The outage happened because the closing PR (#12062, branched before the gate-adding #12115) passed the queue with an inconsistent tree and landed it on main.
Contradictory observations — which is why step 1 is a measurement, not a fix
12:26–13:05Z: five queue entries mass-dequeued without landing while main was red — CONSISTENT with the gate (or the same shared job) failing IN the queue.
Both cannot be true of a static check set. Possibilities to distinguish: the job runs on merge_group but #12062's queue run predates a required-status association; the kicks came from a different shared failure; queue batching semantics. ⛔ Do not wire anything until this is measured.
⚠️ RESOLVED by PR #12338 — and one of the two was not an observation.
12:26–13:05Z was real and is positively identified. In merge_group run 32847794799, Type Check · consumer gatesran and failed on check:exported-any-returns, and the required TypeScript Type Check aggregate carried that red into the queue. CI was green on those same merge groups, ruling out "a different shared failure".
The third candidate — a required-status association postdating the queue run — is ruled out: the Required-Set Patrol reported registry-vs-live agreement in both directions at 04:35Z (before) and 16:35Z (after).
Measure: enumerate which jobs/checks actually execute on merge_group events today (workflow triggers + a real queue run's check list), and diff that against the set of tree-global ratchets that run at PR level (consumer gates, spec-liveness family, the shrink-only ledgers). Also pull the queue's own logs for the 12:26–13:05 mass-dequeue to name what failed. — ✅ DONE, PR ci(lint): record the merge-queue check-set measurement — the queue already runs every tree-global ratchet #12338.
Explicitly out of scope: making ratchets diff-scoped (blinds them — their whole point is global invariants), and auto-re-running checks on all open PRs when a gate lands (expensive; queue parity covers the same hole at one chokepoint).
Why this is the structural layer
Detection was fast (11 min) and the repair was one line; most of the outage was the defect being ABLE to land, plus the fix queuing behind the outage's own wreckage. Queue parity removes the first; the second is a process convention (p0 fix-forward queue-jump / maintainer break-glass merge) that belongs to the PM/skills text, not to CI — noted for the maintainer separately.
⭐ That closing paragraph is now load-bearing on the decision: option A in #12343 (require the queue for main) removes the first half and lengthens the second, because a p0 fix-forward could no longer be hand-merged past the wreckage.
Blocked-by: #12343
Filed by the spec PM seat (
session_01NDGG54XF5gbTLdQzCtnaVV) after the 2026-08-25 stale-ledger outage (~70 min of fleet-wide red; incident cards: the p0 card, its two sibling filings, fix PRs #12186/#12192). Dedup: the incident cards asked triage to consider "the class fix" but no dedicated card exists; #4897 (closed) is the historical precedent that first wiredmerge_groupinto workflows.The mechanism this card closes
The gate that went red (
check:exported-any-returns) is a tree-global ratchet: it judges the whole tree's consistency (ledger vs built dist), not the diff. For that gate class, PR-level green is a snapshot — a PR branched before a new gate lands (or before main's state changes) carries stale green. The only place that sees the REAL merge state is the merge queue. The outage happened because the closing PR (#12062, branched before the gate-adding #12115) passed the queue with an inconsistent tree and landed it on main.Contradictory observations — which is why step 1 is a measurement, not a fix
merge_group.Both cannot be true of a static check set. Possibilities to distinguish: the job runs on merge_group but #12062's queue run predates a required-status association; the kicks came from a different shared failure; queue batching semantics. ⛔ Do not wire anything until this is measured.
Steps
merge_groupevents today (workflow triggers + a real queue run's check list), and diff that against the set of tree-global ratchets that run at PR level (consumer gates, spec-liveness family, the shrink-only ledgers). Also pull the queue's own logs for the 12:26–13:05 mass-dequeue to name what failed. — ✅ DONE, PR ci(lint): record the merge-queue check-set measurement — the queue already runs every tree-global ratchet #12338.merge_groupcoverage (the merge queue 已启用但零 workflow 声明merge_group—— 队列里一个检查都不跑,进队列的 PR 只会空转并锁死自己的分支 #4897 pattern). After that, a fix(client): bind the three verifiable methods of the unannotated return-type erasure population (#11925) #12062-shaped race is KICKED in the queue instead of landing red on main. — ⛔ PREMISE FALSE: none is missing. The gap is settings; see [Decision] The merge queue is already at check-set parity — the 2026-08-25 outage rode a commit that never produced a merge_group build. Two settings toggles, or neither? #12343.Why this is the structural layer
Detection was fast (11 min) and the repair was one line; most of the outage was the defect being ABLE to land, plus the fix queuing behind the outage's own wreckage. Queue parity removes the first; the second is a process convention (p0 fix-forward queue-jump / maintainer break-glass merge) that belongs to the PM/skills text, not to CI — noted for the maintainer separately.