Skip to content

fix(ci): count speculative-stack inheritance as one hit in merge-queue triage - #13107

Merged
os-elon merged 2 commits into
mainfrom
claude/issue-12933-queue-stack-inheritance
Aug 29, 2026
Merged

fix(ci): count speculative-stack inheritance as one hit in merge-queue triage#13107
os-elon merged 2 commits into
mainfrom
claude/issue-12933-queue-stack-inheritance

Conversation

@os-elon

Copy link
Copy Markdown
Collaborator

Fixes#12933

The workflow already parsed the merge-queue branch name and threw away the half
that mattered:

// .github/workflows/merge-queue-triage.yml:146-147 (before)constm=/^gh-readonly-queue\/.+\/pr-(\d+)-[0-9a-f]{40}$/.exec(run.head_branch??'');

One capture group — the PR number. The 40 hex digits were matched but not
captured
, and they are the commit the queue built the PR on top of. Under
GitHub's speculative stacking every PR queued behind a broken one inherits its
tree, so for a deterministic failure the anchor issue's "N distinct pull
requests" measured queue depth, not flakiness — and it kept climbing until
the owner landed a fix. The signal was loudest exactly where it carried the
least information, and every reader had to re-derive that by hand.

The change

  • Capture the base sha into one shared QUEUE_REF constant, used both at the
    entry check and by the new run index.
  • Build runId -> {base, head} from the merge_group runs already fetched
    for the 24 h counts, plus this run. Zero additional API callsrecent
    and run.head_sha were both already in hand, which is what makes this
    affordable in a job whose own header calls itself rate-limit-shaped.
  • Union victims whose queue base equals another victim's queue head, and report
    independent hits beside the raw count.

The trigger is untouched — still a.prs.size >= 2. What changed is the
reporting. Raising the bar to "2 independent hits" would silence a real
deterministic break that has already eaten the whole queue behind it, which is
the case a reader most needs to see.

⚠️ Same-stack is full-sha equality in the strict direction: one victim's base
equals another victim's head. Never a base prefix, never time ordering — both
would fuse genuinely independent hits into one stack, trading this limb's false
positive for a false negative, which is strictly worse. A victim whose run is
not in the window index stays its own group: "cannot prove downstream" reads as
INDEPENDENT, never as same-stack, and the count of those is reported.

The anchor issue now reads 3 pull requests ... 1 independent hit, marks each
row root / inherited, and says why the raw count overstates instead of just
printing a smaller number. The issue TITLE is deliberately unchanged: it is a
second identity channel for finding an open anchor, and a body is the one
channel GitHub is known to rewrite.

Verification

Both gates run the shipped bytes — the harness extracts the script out of
the YAML with a real parser and executes it as actions/github-script does — so
the changed YAML is the subject, not a copy.

beforeafter
check-merge-queue-triage-outcome.mjs61 assertions / 18 scenarios73 / 20
... --self-test78 assertions / 15 mutations89 / 17

New scenario A12 is the recorded incident re-scored: three victims whose
queue bases chain (pr-12851's base IS pr-12843's head; pr-12855's base IS
pr-12851's head) must report as one stack with #12843 as root. New
scenario A13 is the symmetric false-negative guard: three genuinely
unrelated victims must still report as three.

Two ablations, and a new keepGreen field so each states what must not move:

  • M15 reverts the capture group. A12 reds; A13 stays green — the control
    that proves A12's red comes from the stack test and not from the anchor body
    merely changing shape.
  • M16 infers same-stack from co-occurrence instead of base === head. A13
    reds. A12 was predicted to be the control here and was not — it reds
    too, because co-occurrence marks #12843inherited as well, and a stack
    whose every row is inherited names no owner to start from. That the root
    marker is the sharper detector here, not the count, is recorded in the table
    rather than smoothed over.

The naming convention has three independent witnesses, checked on today's tree:
scripts/pm/check-governed-queue-guard.mjs, the workflow's own comment, and a
live measurement — git ls-remote showed
gh-readonly-queue/main/pr-13082-d48929efe794b467c169825ecb68a2d015f09d8d at ref
sha b15d260d1..., and d48929efe7 was origin/main's tip at that moment. The
suffix is the base; the ref's own sha is the head.

What these checks could not exercise: the YAML's action wiring (on:,
permissions:, retries:) and the real API payload shapes. head_sha on an
Actions run object is witnessed in-repo by scripts/pm/ci-failure.mjs:1260
(GET /repos/.../actions/runs/{id} returns head_sha). If that field were ever
absent the index would simply stay empty and every victim would read as
independent — the pre-change behaviour, so the failure direction is safe.

Gate family derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack
(first derivation reported STALE TREE and still exited 0; re-derived after
syncing — identical list). 30 of 31 green at 3466786098;
check-test-completeness.mjs is NOT MEASURED — it takes a turbo test log
argument and answered with its usage line, which is an invocation error, not a
red gate.

skip-changeset derived rather than asserted: pr-automation.yml fires on every
pull request with no paths: filter, changeset-check's only exemptions are
that label and the changesets release PR, and .github/labeler.yml never applies
it automatically. A workflow-plus-scripts diff releases nothing, so the label is
the only available route and is applied on this PR.


Generated by Claude Code

…e triage
The merge-queue-triage workflow already parsed the queue branch name and threw
away the half that matters: `/^gh-readonly-queue\/.+\/pr-(\d+)-[0-9a-f]{40}$/`
matched the 40-hex base commit without capturing it. That sha is the commit the
queue built the PR on top of, so under GitHub's speculative stacking a build
whose base IS another victim's queue head contains that victim's tree by
construction.
Without it, a single deterministic break ejects every PR queued behind it and
the anchor issue reports the queue depth as "N distinct pull requests" -- loudest
in exactly the case carrying the least information.
Capture the sha, index every merge_group run already fetched for the 24h counts
by `runId -> {base, head}`, and union victims whose base equals another's head.
Zero additional API calls: `recent` and `run.head_sha` were both already in hand.
The trigger stays `a.prs.size >= 2`. What changes is the reporting -- the anchor
now says "3 pull requests ... 1 independent hit", marks each row root/inherited,
and explains why the raw count overstates. A victim whose run is not in the
window index stays its own group: "cannot prove downstream" reads as
INDEPENDENT, never as same-stack.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CPrUz21stTFhJRUirdc4yw
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 33237340137 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (1/6) — 失败步骤: Run this shard's tests

    @objectstack/cli:test: FAIL test/serve-publishes-bound-port.e2e.test.ts > #13062 the non-zero half — nothing an ordinary boot publishes may move > publishes exactly the port it was asked for when th
    ↳ 失败原因: @objectstack/cli:test: Error: ENOENT: no such file or directory, open '/tmp/os-bound-port-home-O4vrip/runtime.env_local.json'
    @objectstack/cli:test: FAIL test/serve-publishes-bound-port.e2e.test.ts > #13062 the non-zero half — nothing an ordinary boot publishes may move > follows the DEV AUTO-SHIFT onto the port it really ↳ 失败原因: @objectstack/cli:test: Error: ENOENT: no such file or directory, open '/tmp/os-bound-port-home-cp9OPB/runtime.env_local.json'
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • test/serve-publishes-bound-port.e2e.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 0 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@os-elon
os-elon added this pull request to the merge queueAug 29, 2026
@os-elonClaude

Copy link
Copy Markdown
CollaboratorAuthor

Queue ejection triaged — not this PR's failure. Re-queued once, and this is the one re-run.

domain:devx PM seat (#6023), session session_01CPrUz21stTFhJRUirdc4yw.

The failure, and why it is not ours

Queue build 33237340137 went red on Test Core (1/6), in @objectstack/cli:

FAIL test/serve-publishes-bound-port.e2e.test.ts
Error: ENOENT: no such file or directory,
open '/tmp/os-bound-port-home-O4vrip/runtime.env_local.json'

This PR's diff is two files, read from the diff rather than from the body:

.github/workflows/merge-queue-triage.yml
scripts/check-merge-queue-triage-outcome.mjs

it touches nothing under packages/cli. Control: the failing test file is present on main, so the check that establishes ownership is live rather than answering vacuously.

⚠️And the failure signature is not an assertion. The bot's own triage note makes the distinction that matters here: an AssertionError points at a real behaviour change, while a missing-file / timeout shape points at load, timing or environment. This is ENOENT on a /tmp scratch home the test creates for itself — ⛔ not a verdict about anything this diff does.

What I did, and what I will not do again

Re-queued once. Per this lane's standing rule, a re-run is legitimate only to confirm a failure that names a service the diff does not touch — and at most once in total. This is that one.

If it fails again with the same signature, I will not re-queue a third time. It would then be a real, reproducible failure in @objectstack/cli's e2e suite that this PR happens to be sitting behind, and it belongs to that lane, not to a queue-flake re-roll. "Flake" is not a root cause, and every blind re-queue rebuilds every PR stacked behind this one.

⭐ Worth recording, because the recursion is exact

This is the PR that teaches merge-queue triage to stop counting speculative-stack inheritance as distinct victims — and it was ejected by a failure in a package it does not touch. That is the same family as its own subject: the queue runs the full suite while PR-side CI runs the affected subset, so a queued PR can be reddened by code it never went near, and the raw victim count says nothing about ownership until someone checks the diff by hand. Two seats spent real time deriving exactly that on #12859; this PR is the fix for the count, and it just paid the tax the count hides.

⚠️ Also worth noting for #12933's record: the bot declared its own reading a lower bound"24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界". ⭐ That is an enumeration correctly refusing to report completeness it did not achieve, which is the discipline #12817 is open about. The instrument behaved well; it is the count's meaning this PR repairs.


Generated by Claude Code

Merged via the queue into main with commit 1a540e8Aug 29, 2026
37 checks passed
@os-elon
os-elon deleted the claude/issue-12933-queue-stack-inheritance branch August 29, 2026 07:06
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

2 participants

@os-elon@claude