Uh oh!
There was an error while loading. Please reload this page.
fix(server): stop reaping sessions with live background work - #5690
fix(server): stop reaping sessions with live background work#5690liusqu wants to merge 1 commit into
Conversation
Root cause: the reaper treats a session as idle from `ProviderSessionDirectory.lastSeenAt`, which only advances on turns (`ProviderService.sendTurn` — "a turn is the clearest sign a session is still alive"). But a session keeps working between turns: subagent fleets, workflow runs and background shells outlive the turn that spawned them. A session with no user turn for 30 minutes was `stopSession`'d while its agents were still running, silently killing 30+ minutes of in-flight work. The projection already answers this question. `ThreadBackgroundLiveness` is fed by the same task lifecycle stream for every provider, drops terminal/idle/inert tasks, and is cleared on `session.exited`; its `backgroundLiveness` verdict already rides on the `OrchestrationThreadShell` that the reaper fetches for its `activeTurnId` check. So this is a read of an existing field, not new plumbing. Only "working" earns a reprieve. "monitoring" is watch loops (monitor tasks, background shells) that tick forever by design — honouring it would make those sessions immortal and turn the sweep into a no-op as a leak backstop. Losing a watch loop to the reaper is recoverable; losing a running agent is not. The reprieve is bounded by `backgroundExtensionLimitMultiplier` (48 x the inactivity threshold = 24h by default), so an agent that is wedged yet still emitting progress cannot pin its session open forever. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ApprovabilityVerdict: Needs human review This PR changes session reaper behavior to preserve sessions with active background work for up to 24 hours. While the fix intent is clear and tests are comprehensive, this fundamentally alters when sessions get terminated, affecting resource management and production runtime behavior. Human review recommended for this behavioral change. You can customize Macroscope's approvability policy. Learn more. |
shivamhwp
commented
Aug 26, 2026
gpt-5.6-sol on behalf of shivamhwp. Thank you for working on the session reaper. We are closing this PR because #5677 has already landed the guard that prevents the reaper from killing sessions with live background agents. It updates the same server path and covers the behavior proposed here. The fix is present on main, so keeping this branch open would duplicate completed work. |
Problem
ProviderSessionReapermeasures idleness fromProviderSessionDirectory.lastSeenAt, which only advances on turns (ProviderService.sendTurn). But a Claude session keeps working between turns — subagent fleets, workflow runs, background shells all outlive the turn that spawned them. A session with no user turn for 30 minutes getsstopSession'd while its agents are still running, silently killing the in-flight work.Hit this in production use tonight: a 30+ minute background implementation agent was killed mid-run twice; the only surviving evidence was the worktree state.
Fix
The projection already answers "is native background work still alive":
ThreadBackgroundLivenessis fed by the canonical task lifecycle stream for every provider, drops terminal/idle/inert tasks, and is cleared onsession.exited. ItsbackgroundLivenessverdict already rides on theOrchestrationThreadShellthe reaper fetches for its existingactiveTurnIdcheck — so this is a read of an existing field, not new plumbing. Two files touched.lastSeenAtsessions are spared whilebackgroundLiveness === "working"."monitoring"deliberately does not earn a reprieve: watch loops tick forever by design and would make sessions immortal, turning the sweep into a no-op as a leak backstop. Losing a watch loop is recoverable; losing a running agent is not.backgroundExtensionLimitMultiplier(48× threshold = 24 h by default, configurable like the other two options), so a wedged-but-chatty agent cannot pin its session open forever.activeTurnIdskip is unchanged.Tests
Four new cases in
ProviderSessionReaper.test.ts(idle+working → spared; monitoring → reaped; null → reaped; working past ceiling → reaped). Red-green verified against two deliberately unpatched variants — each new guard is independently load-bearing. Fullsrc/providersuite: 490 passed, 6 skipped. Lint/format clean.🤖 Generated with Claude Code
Note
Medium Risk
Changes when long-idle provider sessions are stopped, which can affect in-flight subagents vs. session leaks; behavior is bounded by a configurable ceiling and covered by new tests.
Overview
Provider session reaper no longer stops sessions that are idle on
lastSeenAtbut still have real background agent work, using the thread shell’s existingbackgroundLivenessfield.After the usual inactivity and
activeTurnIdchecks, the sweep skipsstopSessionwhenbackgroundLiveness === "working"and idle time is below a new cap (inactivityThresholdMs × backgroundExtensionLimitMultiplier, default 48× → 24h)."monitoring"does not get that reprieve (watch loops would otherwise keep sessions open indefinitely).nullor idle past the cap is still reaped, including wedged agents that keep reporting"working".Tests cover working (spared), monitoring/null (reaped), and working past a lowered multiplier ceiling (reaped); the harness can pass
backgroundExtensionLimitMultiplier.Reviewed by Cursor Bugbot for commit 23f9b8b. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Stop reaping provider sessions with live background work
backgroundLivenessis'working'and the idle duration is below a configurable ceiling (inactivityThresholdMs × backgroundExtensionLimitMultiplier, defaulting to 48×).backgroundLivenessof'monitoring'ornullcontinue to be reaped at the normal inactivity threshold.provider.session.reaper.skipped-background-workis emitted when a session is deferred.Macroscope summarized 23f9b8b.