Skip to content

fix(scripts): the required-context self-test proves WIRING with invocations, not prose (#10877) - #10884

Merged
os-zhuang merged 4 commits into
mainfrom
claude/issue-10877-self-test-wiring-presence
Aug 21, 2026
Merged

fix(scripts): the required-context self-test proves WIRING with invocations, not prose (#10877)#10884
os-zhuang merged 4 commits into
mainfrom
claude/issue-10877-self-test-wiring-presence

Conversation

@os-zhuang

@os-zhuangos-zhuang commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Fixes#10877

ℹ️ Built on PR #10878 (#10818), which landed as abe4c6448d while this was in flight. That
PR deleted the other copy of the line filter and added the yaml.parse + shellCommands()
pipeline this fix reads through. It was squash-merged, so origin/main was merged back in
here and the one conflicted file resolved to this branch's version — provably correct, because
main's copy of the file is byte-identical to the PR head this branch started from
(diff exit 0), so "ours" reproduces exactly #10878 + this commit. The diff is now this
change alone: 1 file, +430 / −27.

The defect

--self-test had two presence assertions that decided "is this thing wired up?" from
text a comment, a label or an echo can supply. Presence assertions fail toward a silent
green: the symptom is this gate reporting that a required-context pin runs when it does
not — the #4690 shape one of the assertion messages already names.

Site 1 — the lint job wiring block

A second /^\s*#/ line filter, ~70 lines below the one #10878 repaired, feeding a text match
over a raw slice of lint.yml. Measured on the checked-in file, it was wrong three ways,
all of them false greens about the wiring of a required context:

what it didwhat satisfied it that should not
dropped a line only when its first non-space char is #- name: checkout # run: pnpm check:required-contexts is below
matched any live text in the slicethe invocation spelled in a step name: — prose that survives even a correct comment stripper
sliced '\n lint:''\n typecheck:'typecheck-debt: does not match typecheck:, so the slice covered five jobs (lint, typecheck-source-gates, typecheck-workspace, typecheck-debt, typecheck-consumers) — the pin could migrate out of the required job unnoticed

It also had a false red: /run: pnpm check:required-contexts\b/ needs that literal run of
characters, which a run: | block scalar never contains. Genuinely wiring the pin as one of
several commands would have reddened the gate.

Now a structural read of jobs.lint.steps. None of the three shapes above is a fact about the
job at all.

Site 2 — the caller sweep

callers.join(',') === PATROL_WORKFLOW is two assertions in opposite directions sharing one
wide recognizer
. Measured on required-set-patrol.yml: of its 7 mentions of the flag, 4
are whole-line YAML comments and 3 survive as live text — the genuine
node scripts/check-required-contexts.mjs --verify-required-set, and twoecho steps that
only report on its exit code (the card predicted one). Delete the genuine invocation and either
echo keeps the patrol classified as the standing caller.

Split into the two halves, each on the recognizer its direction needs.

⭐ The asymmetry — why the two halves are NOT unified

Triage made this binding, and the answer is that they must stay separate:

questionrecognizera mention in live prose costs
may this workflow wire the read? (absence)wiresLiveRead, wide — unchangeda false red: loud, names a file, fixed in a minute
does the patrol still run it? (presence)invokesLiveRead, narrow — newa false green: the patrol reads nothing and this gate says it does

Width is not a quality one of them has more of. #10878 was right that wide is correct for the
absence half; the same width is exactly wrong for the presence half. One recognizer serving
both directions is what this file already had
— a /^\s*#/ filter, written twice — and it
was wrong in both directions at once for this reason.

What is shared is the traversal: liveWorkflowText and runCommands are now two readings
of one visitLive walk, so a run: key one reading understands and the other does not cannot
drift. The reading is the part that must not be shared, and invokesLiveRead's docblock says so
with a ⛔ for the next reader.

The disagreement is pinned as an executable fact rather than left to prose: a flag arriving via
env: is wiring for the absence half (#10878's case (h), restated from the other side) and
not an invocation for the presence half.

The reuse decision

check-shard-attestation's invokesScript() was the obvious candidate and is not used:
it hardcodes SCRIPT_BASENAME (parameterising it edits the sibling — the route #10878 already
rejected, and #10628 had to undo one mirrored helper in a neighbouring file), and it splits
commands on /\s+/ then strips quotes off each piece, so it reads echo "pnpm check:required-contexts …"
as running pnpm. commandWords() tracks which words came out of a quoted region instead —
shell does not word-split inside quotes, so that echo is two words, a program and one blob.
⚠️ It is not a third comment stripper: it handles no # at all and runs on shellCommands()
output, where comments are already gone. Only the program word must be unquoted; arguments
may be quoted freely, because refusing "--verify-required-set" would narrow a presence
assertion into a false red.

Evidence

Both limbs throughout. ⭐ A presence test tightened until nothing satisfies it is a worse
defect than the prose it excludes — it shows up as a permanently red self-test, and what a
permanently red self-test gets is loosened. So every prose case is paired with the same wiring
made real, and the checked-in lint.yml and patrol are asserted as live limbs.

origin/main self-test: 124. #10878: 135. This branch: 150.

Reverse verification — two ablations, at the recognizer, no fixture touched. Predicted
counts written down before running; both matched exactly. Each mutation confirmed on disk
by marker counts, never by an editor's exit code, and each restore leg confirmed the same way.

Ablation 1 — the pinWiredIn seam swapped back to the text matcher it replaced
(seam text 1 → 0, ABLATION1 marker 0 → 1). Predicted exactly 6; observed 6, verbatim:

✗ check-required-contexts --self-test — 6 failure(s)
• wiring: a TRAILING YAML comment naming `run: pnpm check:required-contexts` is prose, not wiring (#10877)
• wiring: the pin spelled in a step `name:` is a LABEL, not an invocation (#10877)
• wiring: an `echo` QUOTING the invocation is prose — the quoted blob is one argument, not a program being run (#10877)
• wiring: a TRAILING SHELL comment inside a run: block is prose (#10877)
• wiring: the pin running in a job that is NOT `lint` does not wire `Lint & Repo Gates` (#10877)
• wiring: the pin as one command of a `run: |` block IS wiring — the matcher this replaced could not see it

The real lint.yml presence assertion stayed green under the old matcher — that is the
point: these six isolate exactly the shapes where old and new disagree, and the sixth is the
old matcher's false red.

Ablation 2invokesLiveRead swapped to the wide wiresLiveRead (narrow body 1 → 0,
ABLATION2 marker 0 → 1). Predicted exactly 2; observed 2, verbatim:

✗ check-required-contexts --self-test — 2 failure(s)
• recognizer: a step that only ECHOES --verify-required-set does not INVOKE it — the presence half must not go green on a patrol that reads nothing (#10877)
• recognizer: …and is NOT an invocation for the presence half — the two widths disagree BY DESIGN, and this is the pin that says so (#10877)

Restored from the commit both times (ABLATION1/ABLATION2 markers back to 0, structural
bodies back to 1, worktree clean). No build step is involved — this file is run directly by
node, not resolved through a package dist/.

Gates, all re-run at c20069e17f — the post-merge head — with a clean worktree, exit codes
captured before any pipe (written to a file, not read from a killed pipeline). Derived with
node scripts/pm/dispatch-gates.mjs with no paths — it takes its own change set from the merge
base.

⚠️ Worth recording: check:entry-guard and check:parse-guard are no longer blind spots in
that derivation. They were hand-run on the first pass because dispatch-gates could not reach
them; after #10873 (let a gate declare a scan surface, not just its baseline artifact) landed,
both now derive as ordinary matched families, and both self-tests grew a case for it (47 → 52,
41 → 46). They are still in the table below, now on the derivation's own say-so.

gateits own verdict line
check:required-contexts✓ … --self-test: 150 assertions · ✓ …: 6 required context name(s) pinned across 2 workflow(s); 5 instruction surface(s) scanned
check:cross-package-test-inputsOK: 13 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.
check-ci-filter-parity.mjsOK: all 82 declared cross-package glob(s) (71 unique) are covered…
check:entry-guard✓ …: 129 scripts/ file(s) — every entry guard goes through invoked-as.mjs; 87 export bindings, 77 of them inert on import (10 known-unsafe, ⛔ SHRINK-ONLY)
check:parse-guard✓ …: 128 scripts/ file(s) — every TypeScript parse goes through ts-parse.mjs.
check:nul-bytescheck-nul-bytes: OK (scanned 6296 text file(s) … no raw ASCII control bytes).
check:shard-attestation✓ …: 92 assertions · ✓ …: 2 aggregate gate(s) count 3 declared leg(s) — run because this PR reasons about its lexer; that file is unchanged

npx eslint scripts/check-required-contexts.mjs --no-inline-config — exit 0, no output.

No changeset: scripts/** only, nothing published changes.

Out of scope

The job-level if: is deliberately not judged, and stays that way — lint.yml's own
comment records the reason (four enrolled ci.yml jobs carry a filter-driven if: by design,
#4928, and a skipped job still publishes a context). The re-homed assertion stays step-level,
now over every matching step rather than the first.

Generated by Claude Code


Generated by Claude Code

os-zhuangand others added 3 commits August 21, 2026 15:09
… wiring (#10818)
`uncommentedYaml` dropped a line only when its first non-space character was
`#`, so a TRAILING `# --verify-required-set` on a live line — and a trailing
shell comment inside a `run:` block scalar — survived the strip and reddened
`Lint & Repo Gates` on prose.
A workflow stacks two comment grammars and one line filter was wrong about
both. Each now goes to the thing that knows it: `yaml.parse` for the YAML
layer, and check-shard-attestation's `shellCommands()` lexer — imported, not
re-typed — for the shell inside each `run:`.
Both limbs are pinned in `--self-test`: four prose shapes that must NOT read as
wiring, and six live shapes that must. The trailing-comment pair differs by the
`#` alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
…ations, not prose (#10877)
Two presence assertions in `--self-test` decided "is this wired up?" from text a
comment, a label or an `echo` could supply. Both error toward a silent GREEN —
the gate reporting that a required-context pin runs when it does not.
- the `lint` job block carried a second `/^\s*#/` line filter and matched
`/run: pnpm check:required-contexts\b/` over a raw text slice. It is now a
structural read of `jobs.lint.steps`. The slice was also bounded by
`'\n typecheck:'`, which is not the job after `lint:` — measured, it covered
five jobs.
- the caller sweep's single equality was two assertions in opposite directions
sharing one WIDE recognizer. Split: the absence half keeps `wiresLiveRead`,
the presence half gets `invokesLiveRead`.
The two recognizers are deliberately different widths and are pinned together,
including one case where they disagree by design.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
@claudeclaudeBot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 21, 2026
…lf-test-wiring-presence
# Conflicts:
#	scripts/check-required-contexts.mjs
@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 32500802189 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Console Pin Gate — 失败步骤: Build the Console SPA at the pinned objectui SHA

    ✗ Build failed in 6.03s
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

  • ⚠️本次没有可用的聚合签名(日志里没有能解析出测试文件名的 FAIL 行)—— 这不是「没有同签名的其他 PR」,是这一轮没测到。跨 PR 聚合本次不可用,请手工比对其他 PR 的同类评论。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 30 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Merged via the queue into main with commit 7e75dc7Aug 21, 2026
33 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-10877-self-test-wiring-presence branch August 21, 2026 16:15
os-zhuang pushed a commit that referenced this pull request Aug 21, 2026
…run (#10889)
`invokesScript()` in check-shard-attestation added adjacency for #6589 but
tokenized with `command.split(/\s+/)` plus a per-token quote strip, so a quoted
string handed to `echo` was torn into words and the script name plus its flag
INSIDE that string read as a genuine invocation.
The reading is now the quote-aware one #10884 built for the sibling gate, moved
to the lexer whose output it consumes rather than mirrored: `commandWords` and
`invokes` live beside `shellCommands` in check-shard-attestation, and
check-required-contexts imports them over the one import edge that already
existed between the two files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/mskip-changesetPR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] check-required-contexts' self-test proves its own WIRING with text a comment can supply — two presence assertions that go green on prose

2 participants

@os-zhuang@claude