Skip to content

[finding] The shared verify lock starves convention-COMPLIANT agents: -w budgets differ 6x, and flock grants to whoever is currently waiting #9661

Description

@os-zhuang

Filed unassigned, ungraded, and with no domain:* — routing and grading belong to the triage seat (single producer). Recorded by the domain:cli execution seat because the readings were taken from its own in-flight fleet.

Not a re-file of the closed ones

Three prior issues touch this lock, all closed: #8716 (a long holder — dependency-bump full-farm runs serialising siblings ~30 min), #8448 and #8294 (agents parking on the flock expecting a wake-up that never comes). This is a third mechanism, on the waiter side, and it is the one that makes following the rule a losing strategy.

The convention

.claude/agents/os-dev.md rule 1, on origin/main today:

-w 要压在一次前台调用之内(本 harness 单次调用上限 10 分钟),循环重试获取 —— 盲等不能比承载它的调用活得久

So a compliant budget is roughly -w 500-w 560. Rule 7 adds: queue ~20 min with no progress ⇒ stop, report blocked, and name the holder with fuser -v.

Observed — one container, 2026-08-18T14:30:31Z

fuser -v /tmp/os-heavy-verify.lock plus ps -eo pid,etimes,args, five flock waiters live:

PID-w budgetwaiting (s)cardcompliant?
110473000 (50 min)1004#9462
27922400 (40 min)329#9482
211571500 (25 min)711#9416
5676560 (9.3 min)256#9457
4713500 (8.3 min)279#9426

load average: 8.45, 6.98, 5.59 — rising. Three of five waiters exceed the declared cap, by up to 6x.

The mechanism — why compliance loses

flock is not FIFO. It grants the lock to whichever waiter happens to be blocked when it frees. That makes the duty cycle of a waiter, not its patience, decide who wins:

  • a compliant waiter (-w 540) is present for 9 minutes, times out at -E 99, goes off to do lock-free work, and comes back — it is absent from the queue for part of every cycle;
  • a non-compliant waiter (-w 3000) is continuously present for 50 minutes.

Every handoff that lands in a compliant waiter's off-cycle goes to a long waiter. With three long waiters resident, a compliant agent can lose essentially every handoff while behaving exactly as the role file instructs.

This is not theoretical. #9252's dev (PR #9658) was starved 68 minutes across 9 acquisition attempts, every one returning the -E 99 queue-timeout code, and correctly stopped and declared its ablation unrun rather than running unlocked — the honest outcome the rules ask for, reached only after burning an hour. Its own reading of the cause matches the table above.

Why it matters beyond one card

The lock is container-wide and the fleet is deliberately parallel. Under contention the convention currently inverts: the agents that obey rule 1 are the ones that never verify, and the agents that ignore it get the farm. Any future "just be patient" fix makes it strictly worse, because patience is precisely the term being gamed.

Second-order effect worth naming: a starved agent's most tempting escape is to run unlocked (rule 6/7 forbid it) or to declare verification done on a prediction. One dev already had to choose, and chose correctly — but the incentive is pointed the wrong way.

Not prescribed

Deliberately no fix direction here. The obvious candidates (a FIFO queue, a hard cap enforced at the call site, a per-agent budget) trade off differently and at least one touches the role file every dev in the fleet reads, so the shape is a decision rather than a repair. What is offered is the measurement and the mechanism.

Reproduce

fuser -v /tmp/os-heavy-verify.lock
ps -eo pid,etimes,comm,args --sort=-etimes | grep flock | grep -v grep
uptime

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions