Uh oh!
There was an error while loading. Please reload this page.
fix(orchestrator): Keep Home, steers, and errors honest when work ends - #5459
fix(orchestrator): Keep Home, steers, and errors honest when work ends#5459mwolson wants to merge 348 commits into
Conversation
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
One convention issue found in the new provider-ingest settlement code: a statically known tagged failure is recovered with Effect.catchTag instead of Effect.catchTags.
Posted via Macroscope — Effect Service Conventions
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
ApprovabilityVerdict: Not approved Macroscope's review found this PR not approvable — This is a 964-file Orchestration V2 architecture and persistence migration that adds provider integrations, new workflows, protocol/schema changes, and product-default changes across server, web, and mobile. Unresolved projection and provider-lifecycle race findings add material consistency risk beyond the scope of an auto-approvable fix. Not approved because:
Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more. |
Uh oh!
There was an error while loading. Please reload this page.
5229063 to
6c322c7CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Port pingdotgg#5459 settled-thread lifecycle follow-ups: pin clear on settle, preserve newer updatedAt on activity unsettle, fresher archive membership, settle runs filter, and typed activity-unsettle prep errors.
25de21d to
0af2a6eComparepingdotgg#5459's ProjectionSnapshotQuery.settled test must provide the same ThreadBackgroundLiveness and ThreadPlanProgress layers the live snapshot query now requires on CTM.
0a6ccc1 to
5908b04CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
22bd872 to
a27c1ccCompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Effect service conventions review of the settle-lifecycle changes. The previously flagged Effect.catchTag usage in ProviderEventIngestor.ts is now Effect.catchTags({ ... }), and the new/changed service code (ProjectionStore, EventSink, EffectOutbox, ProviderRuntimeRecoveryService, ProviderEventIngestor, shared settlement helpers) follows the import, layer, and dependency-acquisition rules. Two small change-discipline findings in packages/client-runtime/src/state/threadSettled.ts.
Posted via Macroscope — Effect Service Conventions
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
dae270f to
b27bee8CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
a186d64 to
5b1a115Comparea6e0049 to
50a57b6CompareThe v2 equivalents of main's pingdotgg#8984 and pingdotgg#8922: a "Working for ..." header anchors the active run, the trailing live tool row survives between actions in past tense instead of vanishing, and a shimmering Thinking row marks reasoning gaps. During workspace preparation the header shows "Setting up worktree..." (driven by the local dispatch flag or the v2 run's preparing status, so remote viewers see it too), the composer footer span is gone, and draft promotion waits until the run starts or startup fails instead of navigating mid-preparation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OpenCode 2 now writes an occurrence timestamp when lastError is set or cleared. The same message after a clear is a new occurrence; unchanged text on an uncleared error keeps the original timestamp. The optional lastErrorAt field is added to the provider-session contract so the adapter can persist it without depending on pingdotgg#5459 settled-thread lifecycle.
Stop retrying definitive missing thread snapshots, normalize V2 shell archive membership, preserve provider retry metadata, and harden settled projection ordering, activity unsettle, ingest timestamps, snapshot reads, and Waiting lifecycle invariants.
Settlement-only merge now applies pinnedAt on explicit settle so pins still clear, and activity unsettles no longer rewind updatedAt behind a newer metadata bump.
Add settledOverrideAt on V2 app threads so keep-active and settled pins keep a stable establishment clock across metadata renames. Prefer that field in the activity-unsettle ordering guard, with settledAt/updatedAt fallbacks for pre-field rows, and stamp it on settle, keep-active, and activity clear paths.
Pass settledOverrideAt through the client reduceThreadSettlementEvent path so live sync matches the server pin floor, stamp a new keep-active floor when pin promotes a settled thread, and keep existing active floors when re-pinning.
Re-apply the stranded ChatView and ChatComposer half of pingdotgg#5551 on CTM. Main already gates the Build/Plan toggle behind planModeEnabled, but the v2 ChatView rewrite left the beta setting and Beta panel in place while re-exposing the toggle, /plan and /default, and Shift+Tab. When the flag is off, force the effective interaction mode to default so a thread with a stored plan mode cannot stay trapped after the control is hidden. The next send persists default. Slash commands send as plain text, and the keyboard shortcut is a no-op.
pingdotgg#6123 thread-error-banner-dismiss remembers the remount-safe banner key as thread plus message. After that landed on CTM, dismissing one durable failure hid later identical text for the rest of the app session. Key the session mask with the occurrence timestamp when present so a later run with the same lastError still shows.
The 30-minute idle reaper still tears down the live provider process. It no longer writes lastError null on idle, manual, or server stop. Release reads the latest projected session so an adapter-stamped error survives process teardown.
A repeated process failure must get a new occurrence timestamp so a dismissed banner can reappear. Idle release still preserves the prior timestamp. The web banner key uses lastErrorAt only and no longer falls back to updatedAt.
ae21654 to
57fca87Compare| const nextThread = reduceThreadSettlementEvent({ | ||
| current: currentThread, | ||
| eventType: event.type, | ||
| settlement: { |
There was a problem hiding this comment.
🟡 Mediumorchestration-v2/ProjectionStore.ts:1261
thread.unsettled events lose event.payload.unsettledAt in the durable projection, so shell snapshots report unsettledAt as the previous value (often null) instead of when the thread was explicitly kept active. Include unsettledAt in the settlement merge.
settlement: {
+ unsettledAt: event.payload.unsettledAt ?? null,🤖 Copy this AI Prompt to have your agent fix this:
In file @apps/server/src/orchestration-v2/ProjectionStore.ts around line 1261:
`thread.unsettled` events lose `event.payload.unsettledAt` in the durable projection, so shell snapshots report `unsettledAt` as the previous value (often `null`) instead of when the thread was explicitly kept active. Include `unsettledAt` in the settlement merge.
| // Only a definitive missing thread skips the candidate. Other read | ||
| // failures must surface so callers can retry rather than silently | ||
| // store activity without an unsettle attempt. | ||
| const projectionOption = yield* projectionStore.getThreadProjection(threadId).pipe( |
There was a problem hiding this comment.
🟡 Mediumorchestration-v2/ProviderEventIngestor.ts:356
Archived threads receive a thread.unsettled event for pending non-auth runtime-request.updated activity, clearing their settledOverride instead of leaving archived settlement state unchanged. getThreadProjection returns the still-present archived projection, and this path does not reject it before appending the synthetic event; restrict candidate generation to active threads.
🤖 Copy this AI Prompt to have your agent fix this:
In file @apps/server/src/orchestration-v2/ProviderEventIngestor.ts around line 356:
Archived threads receive a `thread.unsettled` event for pending non-auth `runtime-request.updated` activity, clearing their `settledOverride` instead of leaving archived settlement state unchanged. `getThreadProjection` returns the still-present archived projection, and this path does not reject it before appending the synthetic event; restrict candidate generation to active threads.
Uh oh!
There was an error while loading. Please reload this page.
| if (nextThread === projection.thread) { | ||
| return projection; | ||
| } | ||
| return { ...base, thread: nextThread }; |
There was a problem hiding this comment.
🟡 Mediumstate/orchestrationV2Projection.ts:207
A delayed thread.unsettled rewinds projection.updatedAt to event.occurredAt, even when the projection already has a newer timestamp from a concurrent metadata update. Although reduceThreadSettlementEvent preserves the newer thread.updatedAt, spreading base here still makes client ordering/activity views treat the thread as stale; preserve the later of the existing projection timestamp and event.occurredAt, as the server reducer does.
🤖 Copy this AI Prompt to have your agent fix this:
In file @packages/client-runtime/src/state/orchestrationV2Projection.ts around line 207:
A delayed `thread.unsettled` rewinds `projection.updatedAt` to `event.occurredAt`, even when the projection already has a newer timestamp from a concurrent metadata update. Although `reduceThreadSettlementEvent` preserves the newer `thread.updatedAt`, spreading `base` here still makes client ordering/activity views treat the thread as stale; preserve the later of the existing projection timestamp and `event.occurredAt`, as the server reducer does.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 57fca87. Configure here.
Uh oh!
There was an error while loading. Please reload this page.
| const next = new Set(current); | ||
| next.add(input.context.providerTurnId); | ||
| return next; | ||
| }); |
There was a problem hiding this comment.
Finalize claim races new turns
Medium Severity
finalizeActiveTurn nulls activeTurn and only then adds the id to finalizingTurnIds. startTurn checks those refs separately, so a concurrent start (the handoff timeout is forked) can observe both empty and open a second provider turn while the first is still emitting terminals.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 57fca87. Configure here.
6081201 to
95bacddCompare…d-letters Wire failFromDeadLetter onto the turn-start service and compensate before the outbox fails a start or restart that exhausted retries, matching the v2.1 port.
There was a problem hiding this comment.
Scope note: only apps/web/src/** TS/TSX/CSS changes were reviewed. Findings are the newly added global CSS in index.css (dead/duplicated rules) and the composer context strip's motion contract, which changed lines break while the code that depends on it is retained.
Posted via Macroscope — UI Consistency
| aria-label="Run on" |
There was a problem hiding this comment.
The strip's compact/expand FLIP animation still keys on [data-composer-context-control] (BranchToolbar.tsx:232, consumed by useLabelsOverflow), but this PR removes that attribute from the environment selector, the env-mode selector (locked and select paths) and the branch selector. The only remaining carrier is the vertical Separator at BranchToolbar.tsx:545, so collapsing now animates a 1px separator while the actual controls jump to their new positions. Consider keeping the attribute on the toolbar-mode triggers (it is inert in panel mode), or removing the FLIP pass as well if the motion is intentionally gone.
| aria-label="Run on" | |
| aria-label="Run on" | |
| data-composer-context-control |
Posted via Macroscope — UI Consistency
| .glass-opacity-slider { | ||
| --glass-slider-progress: 0%; | ||
| --glass-slider-fill-offset: 0.5rem; | ||
| --glass-slider-fill-position: calc( | ||
| var(--glass-slider-progress) + var(--glass-slider-fill-offset) | ||
| ); | ||
| appearance: none; | ||
| height: 1.5rem; | ||
| background: transparent; | ||
| cursor: pointer; | ||
| } |
There was a problem hiding this comment.
This whole .glass-opacity-slider block (through the forced-colors rules) is unreferenced — the glass-opacity input in SettingsPanels.tsx:1111 uses settings-slider with --settings-slider-progress, and no literal or composed glass-opacity-slider class exists anywhere in the repo. It is also a verbatim copy of .settings-slider below with only the custom-property names changed, so it adds a second owner for the same slider treatment. Suggest dropping it (or, if a distinct slider is intended, wiring a call site and deriving it from .settings-slider rather than duplicating the vendor pseudo-element rules).
Posted via Macroscope — UI Consistency
| .alert-glass { | ||
| --alert-glass-tint: transparent; | ||
| background: | ||
| linear-gradient( | ||
| color-mix(in srgb, var(--alert-glass-tint) 4%, transparent), | ||
| color-mix(in srgb, var(--alert-glass-tint) 4%, transparent) | ||
| ), | ||
| color-mix(in srgb, var(--background) var(--glass-opacity), transparent) !important; | ||
| -webkit-backdrop-filter: blur(var(--glass-blur)) saturate(var(--glass-saturation)); | ||
| backdrop-filter: blur(var(--glass-blur)) saturate(var(--glass-saturation)); | ||
| } | ||
| .alert-glass[data-variant="error"] { |
There was a problem hiding this comment.
alert-glass is already defined as @utility alert-glass (line 284) and is what ProviderStatusBanner consumes. This second definition in @layer components duplicates it and omits the utility's @supports not ((-webkit-backdrop-filter: blur(1px)) or (backdrop-filter: blur(1px))) opaque fallback; because !important declarations resolve in reverse layer order, this components-layer background: ... !important outranks the utility's important fallback, so browsers without backdrop-filter get a translucent, unblurred banner instead of the solid background. Suggest deleting this block and keeping the @utility as the single owner.
Posted via Macroscope — UI Consistency
| .surface-subheader { | ||
| @apply flex h-10 min-h-10 shrink-0 items-center border-b border-border/60 bg-background; | ||
| } | ||
| [data-preview-panel-mode="inline"] [data-right-panel-surface-content] [data-surface-subheader] { | ||
| height: calc(var(--spacing) * 7); | ||
| min-height: calc(var(--spacing) * 7); | ||
| margin-bottom: calc(var(--spacing) * 3); | ||
| border-bottom-color: transparent; | ||
| } |
There was a problem hiding this comment.
.surface-subheader has no consumer: all four subheaders (DiffPanelShell, FileBrowserPanel, FilePreviewPanel, PreviewChromeRow) still carry the same geometry inline and only set data-surface-subheader. The inline-mode rule below is also redundant — every carrier already has in-data-[preview-panel-mode=inline]:h-7/min-h-7/mb-3/border-b-transparent, and those utilities sort after @layer components, so they win wherever this rule matches. Either apply surface-subheader at those call sites and drop the duplicated utility strings, or drop both rules so the shared geometry keeps one owner.
Posted via Macroscope — UI Consistency
| className="min-w-0 max-w-[240px] truncate transition-[max-width,opacity] duration-300 ease-out group-data-[compact]/composer-context:max-w-0 group-data-[compact]/composer-context:opacity-0" |
There was a problem hiding this comment.
The label previously animated transform/opacity on an inner data-composer-label-motion span with motion-reduce:transform-none motion-reduce:transition-opacity; this replaces it with a max-width transition and no reduced-motion guard, so reduced-motion users now get an animated layout change on every collapse. Same class appears at line 79 and in BranchToolbarEnvModeSelector/BranchToolbarBranchSelector.
| className="min-w-0 max-w-[240px] truncate transition-[max-width,opacity] duration-300 ease-out group-data-[compact]/composer-context:max-w-0 group-data-[compact]/composer-context:opacity-0" | |
| className="min-w-0 max-w-[240px] truncate transition-[max-width,opacity] duration-300 ease-out group-data-[compact]/composer-context:max-w-0 group-data-[compact]/composer-context:opacity-0 motion-reduce:transition-none" |
Posted via Macroscope — UI Consistency
| "hover:!bg-black/[0.055] data-pressed:!bg-black/[0.055] dark:hover:!bg-white/[0.075] dark:data-pressed:!bg-white/[0.075]"; |
There was a problem hiding this comment.
These row classes override Button's core contract at the call site — height (h-9 sm:h-9), radius, padding, and base/hover/pressed background, with ! needed to beat the ghost variant's own hover colors and raw black/white alpha instead of semantic tokens. Since this treatment is now shared by every panel row (select rows, link rows, split halves, icon actions), a named Button size/variant (e.g. size="panel-row" plus a muted-ghost variant) would express it in the primitive and drop the !important fights and the sm: re-statements; contextual width/color could stay here.
Posted via Macroscope — UI Consistency
Macroscope skipped reviewing this pull request. Per-review cost limit exceeded (workspace setting). This review would cost an estimated $250.50, which exceeds your per-review limit of $10.00. The top 3 files driving up this estimate:
Tip To get this pull request reviewed, you can:
|


What Changed
You can steer a turn while the agent is still working. The run could settle anyway, and the later answer vanished. That later text now stays on the thread.
A provider that never started could leave the thread looking alive forever. Exhausted retries now fail the run.
You settle a thread, and Home still treats it like live work. Missing snapshots keep retrying, archived threads drift back, and Waiting keeps refreshing stale rows. Settled now stays settled. A gone thread stops retrying.
You dismiss a thread error, and a later one never appears, or the old one never leaves. Dismissal lasts until a later error. Plan mode off still left the Build/Plan toggle in the composer after the v2 chat rewrite. The toggle is hidden in that case.
Why
Settle meant the useful work was over in the UI while the agent, the outbox, or a later error was still in play. Home, Waiting, and the composer should follow that same rule.
Julius's bounded thread history on current CTM stays as shipped.
UI Changes
Steered output stays in the thread. Failed starts no longer hang as if they were still starting. Home and Waiting settled state is quieter. Thread error banners dismiss until a later error. Build/Plan is hidden when plan mode is disabled.
No before/after screenshots.
Checklist
Note
High Risk
Changes core orchestration paths—settlement merging, command commit guards, Claude turn lifecycle, and process-loss reconciliation—with race-sensitive behavior that can leave runs stuck or drop recovery if callers mishandle
committed: false.Overview
Threads gain
settledOverrideAtso settle/keep-active pins have a stable establishment time that survives renames and is exposed on shells and mobile projections.Settlement is tightened end-to-end: settle is blocked on queued runs, pending runtime requests, and provider background work; archived threads cannot settle/unsettle; pin/unsettle paths maintain the new timestamp; message dispatch clears overrides via a shared helper. Settled/unsettled events are merged through
reduceThreadSettlementEventin memory and in SQL so stale full-thread payloads cannot overwrite title or other metadata. Provider ingest tests document when pending thread-scoped requests should emitthread.unsettledwithout waking keep-active or non-activity updates.Claude steering tracks per-turn steer counts and frame counts, detects steering handoff results before terminalizing, marks in-flight tools interrupted, and bounds silent handoffs with a timeout that fails the turn if no new frames arrive. Turn finalization is claimed up front to avoid duplicate terminals when the handoff fiber races the frame handler; new turns wait while a turn is finalizing.
Recovery adds optional
updatedBeforefilters on outbox cancel/reconcile andexpectedThreadUpdatedAtoncommitCommandso stale recovery writes no-op when the thread has already advanced.Reviewed by Cursor Bugbot for commit 57fca87. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add Orchestration V2 system with provider adapters, persistence, and V2 UI migration
ProviderAdapterV2service contract and driver registry, with replay testkits for deterministic testingEnvironmentThreadStatein threadState.ts replaces legacy thread/page fields withOrchestrationV2ThreadProjectionandThreadHistoryMeta, removingthreadHasOlderTurns; RPC authorization scopes in RpcAuthorization.ts are remapped to V2 method constantsMacroscope summarized 674b0e4.