Uh oh!
There was an error while loading. Please reload this page.
fix(pm): the verify lock's slice backstop stops explaining an ordinary timeout as a clock fault - #10863
Conversation
…inary timeout The slice backstop asserted a specific fault — "the clock this script polls reported none of it" — without ever comparing slice accounting against the clock. A waiter that reached the head of the queue spent its whole budget in slices (18 x 30s = 540s = BUDGET), so `slices_spent >= BUDGET` came true at the bottom of pass 18, one check before `remaining <= 0` at the top of pass 19. The backstop won that tie by position, and the clock-fault sentence became the DEFAULT message for an ordinary full-budget timeout — contradicted by the same run's own 18 progress lines decrementing 510 -> 30 in exact 30s steps. Both acquisition loops now read the clock, test the wall-clock DEADLINE, and only then consult the count-based backstop. The backstop still terminates the loop; it only blames the clock when `clock_disagrees` catches it out, printing both accounts so the claim can be checked instead of believed. `verdict_queue_timeout` gives every ordinary-timeout path one identical sentence, and the 126/127 "flock is gone" branch moves above the bounds so an established cause outranks an inferred one. Self-test grows the falsifying pair: an ordinary full-budget wait must NOT mention a clock, and a genuinely stalled clock must still produce the clock-fault wording. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DdCnBGcHeufjrq7drTD3wt
✅ ACCEPT — reviewer of record: |
| tree | clock | verdict |
|---|---|---|
| pre-fix | healthy | queue-timeout … spent 3s of a 3s budget in flock slices while the clock this script polls **reported none of it** — the card's misfire, verbatim |
| post-fix | healthy | queue-timeout … never acquired · waited 3s · holder pid 21736, held 5s — sleep 20 |
| post-fix | genuinely frozen | lock-unusable … advanced only 0s over the same stretch — **the two accounts disagree** |
The middle row is the fix and the third row is what stops it being a deletion. A guarded message that can never fire is its own defect, and you built the state that fires it (frozen date shim + EPOCHSECONDS/EPOCHREALTIME unset) rather than dropping the wording.
⭐ And the new message prints both accounts, so the accusation is checkable instead of believed. That is the whole point of the card, stated in the artifact rather than in the report.
The rename carries the only real regression risk, and it is clear
queue-timeout → lock-unusable for the stalled-clock case. Verified independently: queue-timeout has zero references anywhere in the repo outside the script itself, and only three files mention os-verify-lock at all — .claude/agents/os-dev.md (governed prose), .github/workflows/lint.yml (0 piping/parsing references; it runs the self-test step), and the script. Nothing parses the verdict kind. Exit 99 unchanged on every path.
The reasoning is right too: under a stalled clock nothing timed out, so queue-timeout was the wrong word for that branch independently of the misattribution.
Both bounded extensions are correct, and the second is the sharper one
- The queue loop's pass-count backstop had the identical defect — asserting a stalled clock from a pass count alone. Leaving it would have returned this card one loop over.
- ⭐ Moving the 126/127 "flock is gone" branch above the two bounds: an established cause must outrank an inferred one. Underneath the backstop, a
flockreturning instantly every pass ran slice accounting to budget in no time and was reported as a stopped clock — a second false cause hiding behind the first. Finding that while fixing the first is the kind of thing that only comes from reading the ordering rather than patching the message.
Ruling on the observation you left me: accept as-is. Do not clamp it.
held -32519s under the frozen-clock fixture, because holder_line subtracts a frozen reading from a real timestamp.
I agree with your reasoning and I am adopting it: the line is reachable only in the state the same sentence declares faulty, and a visibly impossible duration is consistent with "this clock is broken" rather than contradicting it. Clamping to 0 would mean inventing a plausible number inside a message whose entire purpose is checkability — strictly worse than printing an obviously broken one.
holder_line saying the impossible value is deliberate under a stalled clock would cost one line and prevent exactly that. Your call whether it rides along — I am not blocking on it.
Also right
You checked the premise against origin/main first and found the shared checkout at /home/user/objectstack is stale (2d3860df9a, no slices_spent backstop at all) while the card's quoted code is live at 5886ee6d22. That staleness bit me on a different PR an hour ago, where I nearly contradicted a correct agent by grepping the working tree instead of the ref. Checking it up front is the right reflex.
And you did not disturb the live shared lock — private lock files under a temp dir, with the two heavy runs queued through the real entry point (6m04s and 5m26s behind the #10756 and #10782 siblings). That matters: three agents were using this script while you changed it.
Self-test 48 → 56 cases, run unconditionally in CI. CI at your head: 18 green, 0 failing, 3 running.
Flipping ready and arming.
Generated by Claude Code
Uh oh!
There was an error while loading. Please reload this page.
⛔ merge queue 构建失败 — 先分诊，再决定要不要重排队列构建 32493505410 红了。队列跑的是全量套件（PR 侧 CI 只跑 affected 子集）， 失败的 job（日志抽取，best effort）：
跨 PR 相同签名（24h，按失败测试文件聚合）：
历史信号：
分诊清单：
Generated by Claude Code · merge-queue-triage workflow (#4859) |
Fixes#10717
The defect
scripts/pm/os-verify-lock.shprinted a verdict that asserted a specific fault:It never checked the clock before saying that. In the reported run the clock was
working perfectly and the same run's own output proved it — 18 progress lines
decrementing 510 → 30 in exact 30s steps, with the holder's
heldcounterrising in lockstep.
The cause is an ordering race, not a clock fault. A waiter that reaches the head
of the queue spends its entire budget inside the flock-slice loop, so
slices_spent >= BUDGET(18 × 30s = 540s) came true at the bottom of pass 18,one check before
remaining <= 0would have come true at the top of pass 19.The backstop won that tie by position. For a head-of-queue waiter that consumed
its full budget, the clock-fault sentence was not an edge case — it was the
default message for an ordinary timeout.
The refusal itself was always correct. Only the explanation was wrong, and it was
wrong in the most expensive direction: it reads as an infrastructure fault no
retry can fix, when the truth was mundane and actionable — a sibling held the
lock for longer than one full budget. Those two readings lead to opposite next
moves, and the misread has already cost a seat an agent.
The fix — the card's options 1 + 2, composed
Both acquisition loops now read the clock, test the wall-clock DEADLINE, and
only then consult the count-based backstop. The backstop keeps its real job
(terminating a loop whose deadline can never arrive), but it is no longer allowed
to explain the ordinary case.
The clock-fault wording is now gated on
clock_disagrees— the comparison theold sentence claimed to have made and never made:
Scale-free on purpose, so there is no tolerance constant to tune: a stalled clock
reports ~0 whatever the budget, while the benign disagreements (whole-second
truncation at each read, a
flock -w Nreturning a hair early) are a few secondsagainst a budget of hundreds and can never reach half of it. When the two accounts
agree, hitting the backstop means nothing more than "the budget is spent", and it
now says exactly that.
The message that does fire prints both accounts, so the accusation can be
checked instead of believed, and it is relabelled
lock-unusablerather thanqueue-timeout— under a stalled clock nothing timed out; the deadline was neverreachable. Exit code is 99 on every one of these paths, unchanged, and nothing in
the repo parses the verdict kind (grepped: no consumer outside this file).
verdict_queue_timeout()gives every ordinary-timeout path one identicalsentence. That is the point rather than tidiness: a timeout that describes itself
differently depending on which check happened to notice is how this defect
existed at all. The two pre-existing ordinary lines are reproduced byte for byte.
Two bounded in-place extensions, named explicitly
Both are the same defect class as the card (a verdict asserting a cause it never
checked), mechanical, in the same file, under the same gate family:
"the clock this script polls is not advancing" from a pass count alone, never
comparing it against the clock. Left alone, this card comes straight back one
loop over. It now takes the same deadline-first ordering and the same
clock_disagreesgate.126/127"flock is gone" branch moved above the two bounds. A causethis script has actually established must outrank one it infers. Underneath
the backstop, a
flockthat returns instantly every pass ran the sliceaccounting up to BUDGET in no time at all and was reported as a stopped clock —
the same misattribution, one branch over.
Evidence
The falsifying pair, one fixture, pre-fix vs post-fix
The production shape at 1/180 scale: a real holder on a private lock, a healthy
clock, and a waiter whose whole budget is spent at the head of the queue in flock
slices (1 × 3s = 3s budget, exactly as 18 × 30s = 540s).
Pre-fix (
origin/main@5886ee6d22) — an ordinary timeout, blamed on the clock:Post-fix, same fixture:
Post-fix, genuinely stalled clock (frozen
dateshim,EPOCHSECONDSunset) —the guarded wording must still fire, because a message that can never fire is its
own defect:
Static proof the two trees really differ at the emitting statements (not just in
prose):
OLDemitsreported none of itonce andadvanced onlyzero times;NEWemits it zero times andadvanced onlytwice, once per loop. The singlesurviving textual occurrence in
NEWis a comment quoting the old message.Self-test: 48 → 56 cases
Eight new cases, five of them the pair the card asked for, run in CI unconditionally
by the existing
Verify-lock entry-point self-teststep:Gates
Derived with
node scripts/pm/dispatch-gates.mjs(no hand-supplied path list), plusthe two the derivation is structurally blind to. Union run after the final
commit, at
8e830b5a44, under one shared-lock acquisition:check:cross-package-test-inputsAll 90 self-test cases passed./OK: 13 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.check-ci-filter-parity.mjsOK: all 82 declared cross-package glob(s) (71 unique) are covered by 'core' or 'crosspkg' …check-cross-package-test-inputs.mjsOK: 13 package(s) read outside themselves, all declared …check:entry-guard✓ check:entry-guard: 129 scripts/ file(s) — every entry guard goes through invoked-as.mjscheck:parse-guardcheck:nul-bytescheck-nul-bytes: OK (scanned 6271 text file(s) … no raw ASCII control bytes).os-verify-lock.sh --self-test✓ os-verify-lock self-test: all cases pass.(56 cases)No changeset:
scripts/pm/**is repo tooling and publishes nothing, so this PRdeclares no release —
skip-changesetapplied.Note for review
Under the frozen-clock fixture the holder line renders
held -32519s, becauseholder_linesubtracts a frozen reading from a real timestamp. It is reachableonly in the state the very same sentence declares faulty, and a visibly impossible
number is self-falsifying rather than misleading, so it is left alone here rather
than filed or fixed as scope creep. Flagging it in case you read it differently.
Generated by Claude Code