Skip to content

fix(pm): give ci-failure's transport probe the repo-scoped second stage - #10157

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-9966-ci-failure-repo-scoped-probe
Aug 20, 2026
Merged

fix(pm): give ci-failure's transport probe the repo-scoped second stage#10157
os-zhuang merged 2 commits into
mainfrom
claude/issue-9966-ci-failure-repo-scoped-probe

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes#9966

ci-failure.mjs gathered only the account-scoped observation (/rate_limit) and handed it to classifyTransportProbe with no repo-scoped stage, then walked repo-scoped Actions paths through rest(). This adopts the second stage #9963 built — per-caller, which is the shape it was built for.

The container class is real here, and the card's stated consequence is REFUTED — measured before fixing

The dispatch expected this container to refuse every repo-scoped read. It does not. Measured 2026-08-20, one container, seconds apart:

GET /rate_limit -> 200, 14982 left, server: github.com
GET /user -> 200, the real login (os-zhuang)
GET /repos/objectstack-ai/objectstack -> 200, server: github.com (enabled here)
GET /repos/objectstack-ai/objectui -> 403, no server: github.com, no x-ratelimit-*
GET /repos/objectstack-ai/cloud -> 403, same shape
GET /repos/github/docs -> 403, same shape

The refusal is per-repository, not per-session. That does not weaken the card — it sharpens it. A seat pointing this file at a sibling repo (PM_SWEEP_REPO=objectstack-ai/objectui — the cross-repo task CLAUDE.md describes, where a fix spans framework and objectui) is a live specimen, and the account-scoped reading it green-lights on is byte-for-byte the healthy Routine runner's.

What it cost is worse than the card predicted.#9966 expected the exit-3 reading to "degrade into an UNDETERMINED or a raw HTTP number". Measured on origin/main at 2d3860df9:

$ PM_SWEEP_REPO=objectstack-ai/objectui node scripts/pm/ci-failure.mjs --sha 2d3860df9aad...
Error: GET /repos/objectstack-ai/objectui/commits/2d3860df9aad.../check-runs -> HTTP 403
at rest (.../ci-failure.mjs:608:19) [stack trace]
EXIT=1

Node exits 1 on an uncaught exception, and 1 in this file's own exit table is RED"the assertion text was retrieved for EVERY failing check. The output is the answer." A caller branching on $?, which the header instructs it to do, read a transport refusal as a confident verdict about the tree from a container that had not read one byte of it. That is why the fix is a probe stage and not a catch around the walk: exit 3 has to be reached before anything is read, or the answer is only a politer wrong one.

Reverse verification — direction predicted in writing first, then run

legpredictedobserved
--sha, class-4 repo, beforefalsely green probe, then failureprobe reachable, then uncaught throw, exit 1 (worse than predicted — see above)
--sha, class-4 repo, afterexit 3, classified, before the walkexit 3, PREREQUISITE NOT MET — the transport authenticates but repo-scoped reads are refused
--sha, enabled repo, before vs afterunchangedexit 0 both, GREEN — all 29 check-run(s) completed, none failed both
--self-test with stage 2 ablatedclass-4 pin turns red7 failures, exit 1, headline expected "repo-scope-refused" / actual "reachable"

The no-regression leg is not byte-identical, and the reason is not this change: 51 rows on the sha, 22 superseded became 53 rows, 24 superseded because the live board moved between the two runs. The grouped answer (29 check-runs), the verdict and the exit code are identical.

Ablation mechanics. No build or dist/ is involved — this file is executed by path and imports its sibling by relative specifier, so there is no exports resolution that could serve a stale artifact. The mutation was confirmed on disk each way by anchored grep rather than by an editor's exit code: mutated leg ABLATION-9966 = 1 and if (!needsRepoProbe(account)) = 0; restore leg the inverse, plus git diff --stat = 0 lines against the commit.

The ablation also found a defect in the pin itself: with stage 2 removed the 5xx case read unwell.repo.status off a null and crashed, so the harness never printed the class-4 failure it existed to show. A pin that crashes hides its siblings — the assertions are now crash-safe, and that is the second commit.

The design fork the card left open — settled by measurement, not by preference

whether the repo-scoped stage belongs in each caller or whether the two scripts should share one probeTransport

Per-caller, and the deciding evidence is that the two gatherings are already not the same. Stage 1 differs today, on main:

  • check-half-states.mjs re-probes anonymously when first.status !== 200.
  • ci-failure.mjs re-probes when the status is non-200 orrateLimitRemaining === 0.

So "both callers now need the same two stages" is true of stage 2 only. A shared probeTransport would have to pick one of those stage-1 policies and silently change the other file's request pattern — unpinning one of the two self-tests' assumptions to remove a duplication that is one function long. The sequencing rule itself is shared: needsRepoProbe is imported, not re-derived.

The change

  • probeRepoRead() — one GET /repos/{owner}/{repo}. Not GET /user: measured above, /user answers 200 in the refusing container, so an "is this a real endpoint" probe green-lights the very class this stage names. What must be exercised is the scope.
  • probeTransport() fires stage 2 only when stage 1 already returned reachable (needsRepoProbe, imported) — exactly the path that used to green without repo-scoped evidence. The three failing classes short-circuit and cost precisely what they cost before; only the previously-green path spends one core request more.
  • The verdict is produced by re-classifying with the repo reading added, never by patching the stage-1 verdict: one classifier, one place where a verdict is named.
  • The readers are injectable so --self-test drives them offline. It opens no socket, as the header promises.
  • Stage 2 has its own unclassified shape (a repo-scoped 5xx stays null by design), so the probe === null branch now prints the stage-2 reading too — otherwise the reader sees two healthy /rate_limit lines and no explanation of what failed to classify.
  • The fourth container class is added to the header's enumeration with its corrected mechanism, per this file's convention for recording transport surprises.

Self-test

Nine cases added, none rewritten or weakened.selfTest becomes async because the pin drives the gathering — that is where this file's defect lived, and a pin exercising only classifyTransportProbe would restate #9946's self-test instead of covering this file. The added cases pin the verdict, the request sequencing (['rate:token', 'repo']), the short-circuit for failing classes, the 404 and 5xx branches, the tokenless path, and — the regression pin proper — that the same observations with no repo reading still classify as reachable, which is what makes the class-4 fixture a pin rather than a restatement of the fix.

Out of scope, filed not fixed

Neither is addressed here; both remain open.

Gates — run on 9e7aefd, the final commit, each quoted from the gate's own verdict line

node scripts/pm/dispatch-gates.mjs with no args derived the set from the real diff (1 path, merge base 2d3860df9) and named exactly the two families the dispatch listed — no additions.

  • node scripts/check-cross-package-test-inputs.mjsOK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob. (exit 0)
  • node scripts/check-nul-bytes.mjscheck-nul-bytes: OK (scanned 6066 text file(s) -- 6066 tracked, 0 untracked-not-ignored; skipped 5 binary; no raw ASCII control bytes). (exit 0)
  • node scripts/pm/check-half-states.mjs --self-test✓ check-half-states self-test: 469 cases pass. (exit 0) — not path-derived; run because this change now imports needsRepoProbe from it.
  • node scripts/pm/ci-failure.mjs --self-testOK self-test: … the transport probe runs its two stages in order … (exit 0)

Every exit code was captured before any pipe, inside the locked command. The tree was byte-identical to 9e7aefd when they ran.

Labelled skip-changeset: scripts/pm/** is internal PM tooling and publishes nothing (#9963 / #9945 precedent).


Generated by Claude Code

os-zhuangand others added 2 commits August 20, 2026 11:53
…eError
Found by the ablation itself: with stage 2 removed the 5xx case read
`unwell.repo.status` off a null and crashed, so the harness never printed
the class-4 failure it was there to show. A pin that crashes hides its
siblings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@os-zhuangClaude

Copy link
Copy Markdown
ContributorAuthor

Dequeued on CI_FAILURE — diagnosed as not this PR's, re-queuing once to confirm. Signature recorded.

PM seat #6023, session session_01DdCnBGcHeufjrq7drTD3wt, 2026-08-20T12:2xZ (date -u).

⛔ No merge-queue triage comment exists on this PR (0 comments at the time of writing), so there is no ledger entry to consult — this signature is recorded here rather than read from one.

The signature

merge_group run 32368087612, Test Core (1/3), queue ref pr-10157-cc21aad8:

Test Files 1 failed | 413 passed (414) Tests 11035 passed (11035)
Error: Command failed: bash /tmp/sdui-collision-4hPae5/harness.sh
Error: listen EADDRINUSE: address already in use 127.0.0.1:5180
❯ runHarness scripts/gen-sdui-manifest-collision.test.ts:167:15
BUSY_PORT=5180 / PICKED_WITH_BUSY=5180

Why this is not this PR's failure — three independent readings

  1. The diff cannot reach it.git diff --name-only origin/main... returns one path: scripts/pm/ci-failure.mjs. The failing test lives in packages/spec/scripts/ and exercises scripts/gen-sdui-manifest.sh. There is no import, no shared module, no shared fixture.
  2. It passed on this PR's own head. On 9e7aefd: Test Core, Test Core (1/3), (2/3), (3/3) — all four success.
  3. The mechanism is visible in the source and is load-dependent.sdui_pick_free_port (gen-sdui-manifest.sh:200) binds a probe socket, closes it, and only then reports the port free; the caller binds afterwards. Concurrent scanners all start at base=5180 and walk upward deterministically, so they do not diverge — they are handed the same first port. BUSY_PORT == PICKED_WITH_BUSY == 5180 in the log is that race caught in the act.

⇒ ⛔ I am not pushing a change to this PR for it. Widening a one-file PM-tooling PR to patch a port probe in a different package would be the wrong fix in the wrong place, and this PR's own gates were green.

Filed as #10167 with the full diagnosis and a proposed fix shape (hold the socket rather than close-then-report, or give each caller a disjoint base). ⛔ Not graded here — triage's channel.

What I am doing, and the budget for it

Re-queuing once. That is the single sanctioned confirmation run for the "this failure is not mine" case, and it is the only one this PR gets: per the standing rule, "flake" is not a root cause and a second failure would be treated as real. If the same signature returns, ⛔ I will not re-queue again — I will report the blocker rather than keep spending queue slots.

⚠️ ⛔ No merge_pull_request call, at any point, for any reason — landing goes through the queue.


Generated by Claude Code

@os-zhuang
os-zhuang added this pull request to the merge queueAug 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 32368087612 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (1/3) — 失败步骤: Run this shard's tests

    FAIL scripts/gen-sdui-manifest-collision.test.ts [ scripts/gen-sdui-manifest-collision.test.ts ]
    ↳ 失败原因: Error: Command failed: bash /tmp/sdui-collision-4hPae5/harness.sh
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • scripts/gen-sdui-manifest-collision.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 37 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Merged via the queue into main with commit 89f3654Aug 20, 2026
28 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-9966-ci-failure-repo-scoped-probe branch August 20, 2026 12:39
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci-failure.mjs's transport probe has the same false green as #9946 — it classifies from /rate_limit alone, then walks repo-scoped Actions paths

1 participant

@os-zhuang