Skip to content
This repository was archived by the owner on Aug 6, 2026. It is now read-only.

fix(code): unwedge stuck cloud message queues - #2088

Merged
tatoalo merged 2 commits into
mainfrom
posthog-code/fix-cloud-queue-wedge
May 18, 2026
Merged

fix(code): unwedge stuck cloud message queues#2088
tatoalo merged 2 commits into
mainfrom
posthog-code/fix-cloud-queue-wedge

Conversation

@VojtechBartos

Copy link
Copy Markdown
Member

Problem

Users on cloud runs report messages getting permanently stuck "queued", with no automatic recovery. Reproduced from a real session log: SSE streams dropped repeatedly with error: 'terminated', the cloud-task watcher exhausted its 5-attempt reconnect budget, and every subsequent follow-up just sat in the queue. Bug regressed in #1905 (Gate B was added without an SSE-reconnect kick) and #2060 only patched the success path.

What's going on

SessionService.sendCloudPrompt has three queue gates. Two of them could strand a queue forever:

  • Gate B (status !== "connected") — queued the message but never tried to bring the SSE stream back, so no turn_complete ever arrived to trigger a drain. Fix: when status is disconnected or error, fire-and-forget retryCloudTaskWatch(taskId) (already used by the manual "Retry" button). This is the change that actually unwedges the user-reported scenario.
  • Gate A (cloudStatus !== "in_progress") — set isPromptPending: true so the boot-time UI could show a spinner, but the flag was only cleared by turn_complete. A missed turn_complete left the flag stuck and sendQueuedCloudMessages's own !isPromptPending guard then blocked the drain. Fix: drop the eager isPromptPending: true write. The flag now means only "an actual prompt is in flight."
  • handleCloudTaskUpdate status branch — explicitly skipped auto-flush on cloudStatus → in_progress to avoid racing the agent's initial clientConnection.prompt(). That race only exists beforerun_started flips status to "connected". Fix: if a status update with in_progress arrives and session.status === "connected" and the queue is non-empty, schedule a drain. sendQueuedCloudMessages still bails on isPromptPending, preserving the original race protection.

Tests

5 new vitest cases covering each path; full code-app suite (1150 tests) passes.


Created with PostHog Code

Cloud follow-ups got permanently stuck in the local queue when the SSE
watcher exhausted its reconnect budget — Gate B in sendCloudPrompt
queued the message but never restored the SSE stream, so no
turn_complete ever arrived to drain. Two adjacent holes in Gate A and
the cloudStatus handler could also strand a queue on a missed
turn_complete. Each fix is a few lines, all in SessionService.
Generated-By: PostHog Code
Task-Id: 3de9f10b-b668-45c9-8688-eb94b3260be5
@tatoalo
tatoalo marked this pull request as ready for review May 18, 2026 14:36
@greptile-apps

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---### Issue 1 of 1
apps/code/src/renderer/features/sessions/service/service.test.ts:2481-2524
Per the project's "prefer parameterised tests" guideline, these two cases (`"disconnected"` and `"error"`) differ only in the `status` value and the optional `errorMessage` field, so they're natural candidates for a `it.each` table. Keeping them as separate identical test bodies is a DRY violation that will drift over time.
```suggestion it.each([ { status: "disconnected" as const, extraProps: {} }, { status: "error" as const, extraProps: { errorMessage: "Lost connection" }, }, ])( "kicks an SSE retry when queueing on a $status cloud session", async ({ status, extraProps }) => { const service = getSessionService(); mockSessionStoreSetters.getSessionByTaskId.mockReturnValue( createMockSession({ isCloud: true, cloudStatus: "in_progress", status, isPromptPending: false, ...extraProps, }), ); const prompt: ContentBlock[] = [{ type: "text", text: "wake me up" }]; await service.sendPrompt("task-123", prompt); await vi.waitFor(() => { expect(mockTrpcCloudTask.retry.mutate).toHaveBeenCalledWith({ taskId: "task-123", runId: "run-123", }); }); }, );```

Reviews (1): Last reviewed commit: "Merge branch 'main' into posthog-code/fi..." | Re-trigger Greptile

@tatoalotatoalo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to ready and merging to address some customer issues 🚀

@tatoalotatoalo added the Create Release This will trigger a new release label May 18, 2026
@tatoalo
tatoalo enabled auto-merge (squash) May 18, 2026 14:41
@tatoalo
tatoalo merged commit 3a69a85 into mainMay 18, 2026
15 checks passed
@tatoalo
tatoalo deleted the posthog-code/fix-cloud-queue-wedge branch May 18, 2026 14:46
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Create ReleaseThis will trigger a new release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@VojtechBartos@tatoalo