feat(orchestrator): poll builds and stop the ones nothing wants

[behinddwalls]

Summary

Why?

The poll loop was writing the path set on every terminal build — the third concurrent writer on a row where the speculate run, which holds a version across its whole Speculator call, was structurally the one to lose. And with the build stage now start-only, something has to enact cancellation.

What?

The poll loop becomes speculation's kill mechanism. On every poll of a non-terminal build it checks whether anything still wants the build running — batch not halted, the path's entry live and on this build's attempt, the attempt's link naming this very build — and asks the runner to cancel when nothing does. That one level-triggered check subsumes path cancels, batch halts, superseded attempts, and lost dispatch races: no cancel message exists to go stale, and a check that misses one poll is remade on the next.

It cannot cancel a wanted build: every "unwanted" condition is permanent once true, so a stale read only errs toward keeping, and store anomalies (a set, entry, or link that cannot legitimately be missing) also keep the build — a cancel is irreversible. The Cancel call is best-effort so a failure never kills the poll chain that would retry it. The path set is read as that kill list and never written.

Polls now partition on the build ID rather than the batch, and each re-poll mints a distinct message ID so the queue never dedups it away. The halted short-circuit stays removed: a cancelling batch reaches terminal only once its builds stop, and this loop is both what stops them and what watches them stop.

Test Plan

✅ bazel test //submitqueue/orchestrator/... — every unwanted condition cancels; a wanted build never sees a Cancel; anomalies keep the build; a failed Cancel does not fail the poll; statuses recorded per terminal state; and no path-set write happens at all (the set store is wired read-only on the mock).

✅ make fmt, make gazelle

Issues