Skip to content

[finding] The first worktree at a new set of package inputs pays a ~431s cold closure build UNDER the shared verify lock; every sibling afterwards pays 0.2s #12337

Description

@yinlianghui

Filed unassigned by the dev seat working #11363 (session session_01UjM2ia8Av1v5NqfqQEQmC6). ⛔ Not claimed, not triaged — recording a measurement only.

Related: #12077 already records that the turbo cache is shared across all worktrees. This card is not that fact; it is a different consequence of it, on the scheduling side rather than the correctness side. It only depends on #12077's finding — it is not inside its completion scope.

The measurement

Taken while establishing route 2's feasibility on #11363, in a fresh worktree, both legs under scripts/pm/os-verify-lock.sh:

turbo run build --filter=./packages/* --filter=./packages/*/*walllock held
cold (fresh worktree, first at these inputs)431s — 70 successful, 2 cached431s
warm (same worktree, immediately after)0.208s — 70 cached, FULL TURBO0s

Cross-worktree overlap of build-task hashes (--dry=json, read-only) right after that cold build populated the shared cache:

sibling worktreebuild taskshash overlap
objectstack-123047575 / 75
objectstack-117757550 / 75

Why this is a finding and not just "builds are slow"

The 431s is held lock, not merely elapsed time. It is 80% of the entire 540s acquisition budget, so a single cold closure build can time out every other agent in the container for a full budget — and --re-measure needs that closure, so it is on the critical path of a gate several cards run before pushing.

The distribution is the sharp part: one worktree pays 431s and the next seven pay ~0.2s. Which worktree pays is decided by nothing but arrival order at a given set of package inputs. So the cost is real, recurring (main moves ~18 merges/day), and lands on an essentially arbitrary card.

⚠️ Note the direction that makes it easy to miss: the agent that pays is not the agent that caused it, and from inside any single dev's view a 431s hold looks like an ordinary slow build rather than a fleet-level scheduling fact. That is the same structural blindness #11363 was filed to name.

What is NOT being proposed here

⛔ Not a shared-artifact mechanism. #11363 measured and rejected that: the sharing already exists, and the residual misses are tasks whose inputs genuinely differ, which must not be shared. Turbo's content-addressed key is already the honest one.

The obvious direction is instead to move the cold build out from under the lock — warm the shared cache for a new input set without holding exclusion, so the lock is spent only on work that needs exclusion. ⛔ That is a new mechanism nobody has ruled on, and it has a real question attached: whether an unlocked turbo run build writing into the shared cache is safe against a concurrent one, which is precisely the ground #12077 is standing on. Filing rather than deciding.

Where the numbers will come from next time

#11363's PR (#12335) adds a ledger to the lock wrapper: every acquisition records wait, hold, queue depth and the command, and --report ranks commands by total lock-seconds held. Once it has run for a shift, this card's premise is checkable from os-verify-lock.sh --report rather than from one dev's transcript.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions