fix(dispatch-capacity): degrade state when a placed slot outlasts agentHoldTimeoutMs (#419)

[kjgbot]

Closes#419.

The defect

dispatchCapacity had two hold bounds surfaced and neither applied to the failure that actually happens.

• waitWarnMs (30 min) concerns queue WAIT time and never fires with waiting: 0.
• agentlessHoldTimeoutMs (30 min) bounds a slot held WITHOUT a placed agent. A slot with placedAgents: 1 was exempt.

The reaper for occupied slots did exist (agentHoldTimeoutMs, 4 h default) but was never exposed next to the agentless one and never fed the state derivation. So an operator reading /healthz saw dispatchCapacity.state: "healthy" while two occupants had held their slots 13.5 hours — 27× the agentless timeout, 3.4× the placed-agent timeout — with batchSize: 2 and both slots consumed. Total dispatch stop, green subsystem.

The fix

1. Publish agentHoldTimeoutMs on FactoryDispatchCapacityStatus and FactoryPublicDispatchCapacityHealth so a reader can check slotHeldForMs / heldForMs against both reap deadlines.
2. Add the wedge shape the state derivation was missing: occupiedOccupants (aggregate) and per-occupant pastOccupiedDeadline. Mirrors the discipline of agentlessOccupants / pastReapDeadline. Anchored on heldForMs (the clock the reaper uses) so a placement whose spawn took hours is not confused with a placement that outran its deadline. >=, not >: agrees with the reaper's nowMs < dueAtMs skip boundary.
3. Extend deriveDispatchCapacityState / dispatchCapacityState to degrade to stalled when either wedge shape is present, and thread the same signal through normalizePublicHealth so an over-the-wire record without the aggregate count still projects the wedge (mirror of the fix(factory): arm the never-placed reaper independently of startup fleet health (#315) #318 discipline).
4. dispatchCapacity was already in DISPATCH_GATING_SUBSYSTEMS and the top-level status computation already checks it. The state change flips status: "ok" → "degraded" and adds "dispatchCapacity" to degradedSubsystems. ok (container-liveness) is deliberately left alone — recycling a wedged Factory destroys the durable evidence of the wedge and re-imports the lock into the replacement (the Dispatch is permanently wedged: one agent-less lifecycle holds the only batch slot and cannot be reaped, so every queued row spins at 1 Hz forever #303 doc-comment discipline).

Reclamation across a restart

Already survives: heldSinceAtMs rides on the durable lifecycle, inFlightRecordFromLifecycle re-derives it on rehydration, and the finally block at the end of the adopt path calls #rescheduleHeldAgentDeadlineSweep even if a broker call above it threw. The existing agentHoldTimeoutMs reaper (#sweepHeldAgentDeadlines) fires immediately post-boot for any record already past deadline (#412 comment thread confirms the 2026-08-31T11:14Z restart cleared the stranded slots). This PR is the missing observational half: without it, the reaper's absence in a running daemon is invisible on /healthz.

Ablation

Three tests in src/orchestrator/public-health.test.ts:

• Must-fire (dispatchCapacity reports healthy while both slots are held 13.5h by offline agents: no max hold for an occupied slot #419): a 13.5-hour placed-agent occupant (the exact observed shape) → state: 'stalled', occupiedOccupants: 2, degradedSubsystems: ['dispatchCapacity'], status: 'degraded'. ok stays true.
• Must-not-fire (dispatchCapacity reports healthy while both slots are held 13.5h by offline agents: no max hold for an occupied slot #419): a placed-agent occupant one millisecond inside the deadline → state: 'healthy', no occupiedOccupants, no pastOccupiedDeadline, top-level status: 'ok'. slotHeldForMs is deliberately far past the bound to prove the diagnostic anchors on heldForMs.
• Wire re-derivation: a producer that publishes occupants without the aggregate count still projects the wedge in normalizePublicHealth — mirror of the fix(factory): arm the never-placed reaper independently of startup fleet health (#315) #318 fix for the agentless shape.

✓ dispatch capacity health (#303) > degrades when a placed-agent occupant is past agentHoldTimeoutMs (#419)
✓ dispatch capacity health (#303) > leaves a placed-agent occupant within agentHoldTimeoutMs untouched (#419)
✓ dispatch capacity health (#303) > normalizes a placed-agent wedge over the wire without the aggregate count (#419)

Full suite: 750/750 pass across orchestrator/public-health, orchestrator/factory, config/schema, cli/diagnose.

Overlap with related issues

• [factory] Orphan-reaper dispatchCallActive guard can never fire #331 (orphan-reaper dispatchCallActive guard can never fire): distinct. That is a composed-key bug in the #dispatchInFlight lookup path — the reaper reads a bare issue key while writers compose ${issue}:${dryRun}:${phase}. Unrelated to this fix; not addressed here.
• [factory] A hang is not a failure: readinessReconcile can never trip its own breaker, so a wedged sweep is a permanent absorbing state #372 (wedged sweep is a permanent absorbing state): distinct subsystem (readinessReconcile).
• Dispatch is permanently wedged: one agent-less lifecycle holds the only batch slot and cannot be reaped, so every queued row spins at 1 Hz forever #303 / Agent-less lifecycle holds a batch slot past agentlessHoldTimeoutMs on 0.1.68, and dispatchCapacity still reports healthy #315 / fix(factory): arm the never-placed reaper independently of startup fleet health (#315) #318: the agentless-shape antecedents this PR mirrors for the placed-agent shape.

Test plan

• vitest run src/orchestrator/public-health.test.ts — 58/58 pass, incl. 3 new dispatchCapacity reports healthy while both slots are held 13.5h by offline agents: no max hold for an occupied slot #419 tests
• vitest run src/orchestrator/factory.test.ts src/config/schema.test.ts src/cli/diagnose.test.ts — 750/750 pass
• CI green per-workflow at the head sha

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 022020e6-9490-420e-ad06-3554bddee387

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

@coderabbitai review

Requested for exact head c4b032d18ec89eb87f8c633ace1f31c81fc7dbf4.

[cubic-dev-ai]

All reported issues were addressed across 4 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[github-actions]

@coderabbitai review

Requested for exact head f81046735e6bef78f7ac392659cafdbaf9f64911.