Uh oh!
There was an error while loading. Please reload this page.
feat: configurable client-side message queue for useChat - #900
Merged
Conversation
Contributor
🚀 Changeset Version Preview7 package(s) bumped directly, 3 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
View your CI Pipeline Execution ↗ for commit 6c767cd
☁️ Nx Cloud last updated this comment at |
Contributor
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds default queueing for ChangesMessage Queueing Feature
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
@tanstack/ai@tanstack/ai-acp@tanstack/ai-angular@tanstack/ai-anthropic@tanstack/ai-bedrock@tanstack/ai-claude-code@tanstack/ai-client@tanstack/ai-code-mode@tanstack/ai-code-mode-skills@tanstack/ai-codex@tanstack/ai-devtools-core@tanstack/ai-elevenlabs@tanstack/ai-event-client@tanstack/ai-fal@tanstack/ai-gemini@tanstack/ai-grok@tanstack/ai-grok-build@tanstack/ai-groq@tanstack/ai-isolate-cloudflare@tanstack/ai-isolate-node@tanstack/ai-isolate-quickjs@tanstack/ai-mcp@tanstack/ai-mistral@tanstack/ai-ollama@tanstack/ai-openai@tanstack/ai-opencode@tanstack/ai-openrouter@tanstack/ai-preact@tanstack/ai-react@tanstack/ai-react-ui@tanstack/ai-sandbox@tanstack/ai-sandbox-cloudflare@tanstack/ai-sandbox-daytona@tanstack/ai-sandbox-docker@tanstack/ai-sandbox-local-process@tanstack/ai-sandbox-sprites@tanstack/ai-sandbox-vercel@tanstack/ai-solid@tanstack/ai-solid-ui@tanstack/ai-svelte@tanstack/ai-utils@tanstack/ai-vue@tanstack/ai-vue-ui@tanstack/openai-base@tanstack/preact-ai-devtools@tanstack/react-ai-devtools@tanstack/solid-ai-devtoolscommit: |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
packages/ai-solid/src/use-chat.ts (1)
141-144: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winSame queue-sync gap as the React adapter.
options.queueis only captured once when the client is created; there's nocreateEffectre-applying it viaclient().updateOptions(...)the waybody/forwardedProps/contextare synced at Line 152-165. See the parallel comment onpackages/ai-react/src/use-chat.ts(Line 162-168) — same underlying question of whetherqueueis meant to be static-per-session or live-updatable.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-solid/src/use-chat.ts` around lines 141 - 144, The chat queue is only read once when the client is created in useChat, so later changes to options.queue are never synced into the live client. Add a createEffect alongside the existing body/forwardedProps/context syncing that calls client().updateOptions(...) whenever options.queue changes, using the same pattern as the other reactive option updates in useChat so the queue stays in sync if it is meant to be live-updatable.
🧹 Nitpick comments (3)
packages/ai-preact/src/use-chat.ts (1)
157-163: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value
queueconfig isn't kept in sync after mount.Unlike
body/forwardedProps/context(synced via theupdateOptionseffect below),queueis only read once at client construction (useMemokeyed on[clientId]). If a consumer changesoptions.queue(e.g. togglingwhenBusy) after mount without changingid, the change is silently ignored. This mirrors the existing behavior for other one-time options liketools/persistence, so it's consistent with current design, but worth confirming it's intentional for a config surface consumers may want to adjust dynamically (e.g., per-messagewhenBusyoverrides already exist for that case).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-preact/src/use-chat.ts` around lines 157 - 163, The `queue` option in `useChat` is only applied when the client is created in the `useMemo` block, so later changes to `options.queue` are ignored after mount. Update the `updateOptions` path in `use-chat.ts` (the effect that already keeps `body`, `forwardedProps`, and `context` in sync) to also propagate `queue` when `optionsRef.current.queue` changes, using the same `activeClientRef.current === instance` guard as `onQueueChange`. If `queue` is meant to stay one-time like `tools`/`persistence`, make that behavior explicit in the option handling and docs.packages/ai-client/src/chat-client.ts (1)
803-846: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the new
sendOptionsparameter.The extensive JSDoc/example block for
sendMessagewasn't updated to mention the new third parameter (sendOptions.whenBusy), even though it's a public API addition central to this PR.📝 Suggested doc addition
* `@param` body - Optional body parameters to merge with the client's base body for this request. * Uses shallow merge with per-message body taking priority. + * `@param` sendOptions - Optional per-call overrides, e.g. `{ whenBusy: 'interrupt' }` to+ * override the configured queue policy for this one send. * * `@example`🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-client/src/chat-client.ts` around lines 803 - 846, The public JSDoc for sendMessage is missing the new third parameter, so update the documentation block for sendMessage to describe sendOptions and its whenBusy option alongside content and body. Add a brief `@param` entry and, if helpful, a small example showing how sendOptions.whenBusy is used, so the docs match the new signature in chat-client.ts.packages/ai-client/tests/chat-client-queue.test.ts (1)
71-328: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd coverage for
interrupt, per-callwhenBusyoverride, andmaxSize/onOverflow.This suite covers default queueing, drop, FIFO/batch drain, and flush-on-stop/error well, but three documented behaviors have zero test coverage:
whenBusy: 'interrupt'(abort + send now)- per-call
sendOptions.whenBusyoverride onsendMessagemaxSizewithonOverflow: 'reject'vs'drop-oldest'Adding tests for the overflow behavior in particular would surface the
maxSize: 0boundary issue flagged inchat-client.ts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-client/tests/chat-client-queue.test.ts` around lines 71 - 328, Add tests in ChatClient message queue coverage for the missing documented behaviors: verify `whenBusy: 'interrupt'` aborts the in-flight send and allows the new message to send immediately, verify `sendMessage` honors a per-call `sendOptions.whenBusy` override over the client default, and verify `maxSize` with `onOverflow: 'reject'` and `'drop-oldest'` behaves correctly. Use the existing `ChatClient`, `sendMessage`, `getQueue`, `getMessages`, and the held connection helpers as reference points, and add a boundary case for `maxSize: 0` to cover the overflow path mentioned in `chat-client.ts`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai-client/src/chat-client.ts`:
- Around line 905-923: The enqueueMessage queue limit check currently lets one
item through when queueConfig.maxSize is 0 and onOverflow is not 'reject',
because it shifts an empty queue and then still pushes a new message. Update
enqueueMessage to treat maxSize: 0 as a hard cap in the same way as other
full-queue cases, using the existing messageQueue, queueConfig, and onOverflow
logic so no item is enqueued when the limit is zero.
- Around line 882-923: The queued item id is generated twice, so the
`pending.id` seen by `QueueStrategy` does not match the actual entry stored by
`enqueueMessage`. Generate one queued id in `decideWhenBusy`, pass it through to
`enqueueMessage`, and use that same id when building the `pending` context and
when pushing the message so `getQueue()`, `onQueueChange`, and
`cancelQueued(id)` all refer to the same item.
In `@packages/ai-svelte/src/types.ts`:
- Around line 149-152: The `queue` property in the chat state interface is
missing `readonly`, unlike the other reactive getters and the Svelte/Vue
equivalents. Update the `queue` declaration in `types.ts` to be readonly
alongside `messages`, `isLoading`, `error`, `status`, `isSubscribed`,
`connectionStatus`, and `sessionGenerating`, so consumers know it is getter-only
and cannot be assigned to directly.
---
Duplicate comments:
In `@packages/ai-solid/src/use-chat.ts`:
- Around line 141-144: The chat queue is only read once when the client is
created in useChat, so later changes to options.queue are never synced into the
live client. Add a createEffect alongside the existing
body/forwardedProps/context syncing that calls client().updateOptions(...)
whenever options.queue changes, using the same pattern as the other reactive
option updates in useChat so the queue stays in sync if it is meant to be
live-updatable.
---
Nitpick comments:
In `@packages/ai-client/src/chat-client.ts`:
- Around line 803-846: The public JSDoc for sendMessage is missing the new third
parameter, so update the documentation block for sendMessage to describe
sendOptions and its whenBusy option alongside content and body. Add a brief
`@param` entry and, if helpful, a small example showing how sendOptions.whenBusy
is used, so the docs match the new signature in chat-client.ts.
In `@packages/ai-client/tests/chat-client-queue.test.ts`:
- Around line 71-328: Add tests in ChatClient message queue coverage for the
missing documented behaviors: verify `whenBusy: 'interrupt'` aborts the
in-flight send and allows the new message to send immediately, verify
`sendMessage` honors a per-call `sendOptions.whenBusy` override over the client
default, and verify `maxSize` with `onOverflow: 'reject'` and `'drop-oldest'`
behaves correctly. Use the existing `ChatClient`, `sendMessage`, `getQueue`,
`getMessages`, and the held connection helpers as reference points, and add a
boundary case for `maxSize: 0` to cover the overflow path mentioned in
`chat-client.ts`.
In `@packages/ai-preact/src/use-chat.ts`:
- Around line 157-163: The `queue` option in `useChat` is only applied when the
client is created in the `useMemo` block, so later changes to `options.queue`
are ignored after mount. Update the `updateOptions` path in `use-chat.ts` (the
effect that already keeps `body`, `forwardedProps`, and `context` in sync) to
also propagate `queue` when `optionsRef.current.queue` changes, using the same
`activeClientRef.current === instance` guard as `onQueueChange`. If `queue` is
meant to stay one-time like `tools`/`persistence`, make that behavior explicit
in the option handling and docs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 384b81cf-49eb-43ca-a54f-13b3dc63cefd
📒 Files selected for processing (28)
.changeset/message-queue.mddocs/chat/streaming.mddocs/config.jsonpackages/ai-client/src/chat-client.tspackages/ai-client/src/devtools.tspackages/ai-client/src/index.tspackages/ai-client/src/types.tspackages/ai-client/tests/chat-client-queue.test.tspackages/ai-client/tests/chat-client.test.tspackages/ai-preact/src/types.tspackages/ai-preact/src/use-chat.tspackages/ai-preact/tests/use-chat.test.tspackages/ai-react/src/types.tspackages/ai-react/src/use-chat.tspackages/ai-react/tests/use-chat.test.tspackages/ai-solid/src/types.tspackages/ai-solid/src/use-chat.tspackages/ai-solid/tests/use-chat.test.tspackages/ai-svelte/src/create-chat.svelte.tspackages/ai-svelte/src/types.tspackages/ai-vue/src/types.tspackages/ai-vue/src/use-chat.tspackages/ai-vue/tests/use-chat.test.tspackages/ai/skills/ai-core/chat-experience/SKILL.mdtesting/e2e/fixtures/queue/basic.jsontesting/e2e/src/components/ChatUI.tsxtesting/e2e/src/routes/$provider/$feature.tsxtesting/e2e/tests/queue.spec.ts
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Comment on lines
+149
to
+152
| /** | ||
| * Pending messages queued while a stream is in flight. | ||
| */ | ||
| queue: Array<QueuedMessage> |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Mark queue as readonly for consistency.
Every other reactive getter in this interface (messages, isLoading, error, status, isSubscribed, connectionStatus, sessionGenerating) is declared readonly, and the Svelte implementation only exposes queue via a getter (no setter). The Vue equivalent is also wrapped in Readonly<...>. Missing readonly here lets consumers believe chat.queue = [...] is a valid assignment.
♻️ Proposed fix
/**
* Pending messages queued while a stream is in flight.
*/
- queue: Array<QueuedMessage>+ readonly queue: Array<QueuedMessage>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| /** | |
| *Pendingmessagesqueuedwhileastreamisinflight. | |
| */ | |
| queue: Array<QueuedMessage> | |
| /** | |
| *Pendingmessagesqueuedwhileastreamisinflight. | |
| */ | |
| readonly queue: Array<QueuedMessage> |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-svelte/src/types.ts` around lines 149 - 152, The `queue` property
in the chat state interface is missing `readonly`, unlike the other reactive
getters and the Svelte/Vue equivalents. Update the `queue` declaration in
`types.ts` to be readonly alongside `messages`, `isLoading`, `error`, `status`,
`isSubscribed`, `connectionStatus`, and `sessionGenerating`, so consumers know
it is getter-only and cannot be assigned to directly.
…unsubscribe Wires the queue drain into streamResponse's settle path so queued sends actually go out (fifo one-at-a-time, or batch merged with newlines), and flushes any pending queue on stop()/clear()/unsubscribe() so callers don't get a surprise send after tearing down. Also surfaces the queue on the devtools snapshot.
A RUN_ERROR or thrown non-abort error left streamCompletedSuccessfully false, so the finally block's queue handling (guarded by that flag) was skipped entirely. Queued messages were stranded until a later direct sendMessage sent first and drained the stale queued item afterward, inverting message order. Flush (not drain) the queue on the non-success settle path, matching stop()'s existing behavior.
Mirrors the ai-react implementation (17bb417): ChatClient's queue-by-default behavior is now wired through the Preact useChat hook via onQueueChange, with cancelQueued and per-send whenBusy passthrough. Updates the two tests that previously asserted a mid-stream second sendMessage was dropped to instead assert both messages land in order.
The framework packages re-exported QueuedMessage/WhenBusy/QueueConfig/ QueueStrategy/QueueOption from their internal types module but never forwarded them through the public index entry, so importing them from e.g. @tanstack/ai-react failed at the package boundary.
Side-by-side panels (queue/fifo, interrupt, queue/batch) driven by a shared composer, showcasing how each queue strategy handles a message sent while a stream is in flight, with live queue rendering + cancel.
FIFO was stranding user messages when concurrent sendMessage calls both passed the isLoading check (or when drain re-entered sendMessage and re-enqueued items). Claim the client with sendInFlight, walk the queue in a drain loop via deliverMessage, and stop the loop cleanly on interrupt. Also: stable strategy pending ids, maxSize validation, Angular injectChat parity, reload flushes queue, continuation errors still drain, queue option live-updates, docs/skill accuracy, and unit coverage for the new policy branches.
tombeckenhamforce-pushed
the
feat/client-message-queue
branch
from
July 17, 2026 05:49
7401f07 to
83d5407CompareClear @typescript-eslint/no-unnecessary-condition findings in drainQueue and continuation handling, drop an unnecessary type assertion in queue tests, and fix import order / redundant narrowing in the queueing example.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/ts-react-chat/src/routes/queueing.tsx`:
- Around line 6-7: Reorder the imports from '`@/lib/model-selection`' in
queueing.tsx so the type-only ModelOption import appears before the value import
of DEFAULT_MODEL_OPTION and MODEL_OPTIONS, satisfying the configured
import/order rule.
In `@packages/ai-client/src/chat-client.ts`:
- Around line 1655-1669: In the queue-draining loop around deliverMessage,
retain the runtime checks of this.isLoading and this.stopMessageQueueDrain
before and after the await. Add narrowly scoped ESLint suppressions for the
static-analysis warnings on those asynchronous guard conditions, rather than
removing or restructuring the guards.
- Around line 1286-1290: Remove the unnecessary isLoading condition around
drainQueue in the continuation failure catch block, and invoke this.drainQueue()
directly because the stream has already settled there.
- Around line 906-919: Update streamResponse() so the queued-message drain runs
again after sendInFlight is reset to false, including the completion path shown
around deliverMessage(). Ensure the follow-up drain occurs after releasing the
send claim so sends enqueued during the completion window are processed.
- Around line 1645-1652: Update the batch-draining branch in the queue drain
method to continue processing until messageQueue is empty, including messages
enqueued during deliverMessage. Preserve the existing merge and delivery
behavior for each batch, but replace the one-shot return flow with a loop or
equivalent continuation that emits queue changes and drains all pending batches.
In `@packages/ai-client/tests/chat-client-queue.test.ts`:
- Around line 1-6: Move the chat-client queue unit test from the tests directory
to sit alongside chat-client.ts, preserving its existing coverage and updating
relative imports such as ChatClient, normalizeQueueOption, createTextChunks, and
connection adapter types for the new location.
- Around line 521-525: Replace the tautological assertion in the rapid-send test
around client.getQueue() with an assertion that verifies the expected queued
message count while the first user message is in flight. Ensure the check
detects sends bypassing the queue and preserves the intended one-in-flight,
remaining-messages-queued behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: a3e23dc9-8eb6-4aea-8a73-784ccf39bff8
📒 Files selected for processing (15)
.changeset/message-queue.mddocs/chat/streaming.mddocs/config.jsonexamples/ts-react-chat/src/components/Header.tsxexamples/ts-react-chat/src/routeTree.gen.tsexamples/ts-react-chat/src/routes/queueing.tsxpackages/ai-angular/src/index.tspackages/ai-angular/src/inject-chat.tspackages/ai-angular/src/types.tspackages/ai-client/src/chat-client.tspackages/ai-client/src/devtools.tspackages/ai-client/src/index.tspackages/ai-client/src/types.tspackages/ai-client/tests/chat-client-queue.test.tspackages/ai-client/tests/chat-client.test.ts
🚧 Files skipped from review as they are similar to previous changes (8)
- examples/ts-react-chat/src/components/Header.tsx
- packages/ai-client/src/index.ts
- .changeset/message-queue.md
- packages/ai-client/tests/chat-client.test.ts
- packages/ai-client/src/devtools.ts
- docs/chat/streaming.md
- packages/ai-client/src/types.ts
- examples/ts-react-chat/src/routeTree.gen.ts
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Comment on lines
+1
to
+6
| import { describe, expect, it, vi } from 'vitest' | ||
| import { EventType } from '@tanstack/ai/client' | ||
| import { ChatClient, normalizeQueueOption } from '../src/chat-client' | ||
| import { createTextChunks } from './test-utils' | ||
| import type { ConnectConnectionAdapter } from '../src/connection-adapters' | ||
| import type { StreamChunk } from '@tanstack/ai/client' |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Move this unit test alongside chat-client.ts.
This new test belongs beside packages/ai-client/src/chat-client.ts, with its relative imports adjusted.
As per coding guidelines, *.test.ts files must be placed alongside the source they cover.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-client/tests/chat-client-queue.test.ts` around lines 1 - 6, Move
the chat-client queue unit test from the tests directory to sit alongside
chat-client.ts, preserving its existing coverage and updating relative imports
such as ChatClient, normalizeQueueOption, createTextChunks, and connection
adapter types for the new location.
Source: Coding guidelines
Uh oh!
There was an error while loading. Please reload this page.
Loop batch drain so mid-stream enqueues are not stranded, harden deliverMessage with a claim that hands off to isLoading, unify QueueStrategy on WhenBusy + busyReason, and re-export SendMessageOptions. Docs now say drain only on successful settle; tests cover body, mid-drain error/interrupt, and batch re-enqueue.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai-client/tests/chat-client-queue.test.ts`:
- Around line 786-842: Update the “interrupt during FIFO drain” test around the
second connection attempt to track its cancellation, such as by observing the
async connection’s abort signal or cancellation callback. After sending “urgent”
with whenBusy: 'interrupt', assert that the active “second” request was aborted
while preserving the existing ordering and queued “third” assertions.
In `@packages/ai/skills/ai-core/chat-experience/SKILL.md`:
- Around line 445-448: Clarify the “Drain vs flush” documentation to distinguish
interrupt-triggered stream aborts from other active-generation aborts: preserve
queued messages across `interrupt`, while discarding them for stream errors,
non-interrupt aborts, `stop()`, `clear()`, `unsubscribe()`, and `reload()`.
Retain the existing rule that only a successful settle auto-sends queued
messages.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 98982007-a845-456f-8888-a73fef7ae114
📒 Files selected for processing (24)
docs/chat/streaming.mdpackages/ai-angular/src/index.tspackages/ai-angular/src/inject-chat.tspackages/ai-angular/src/types.tspackages/ai-client/src/chat-client.tspackages/ai-client/src/index.tspackages/ai-client/src/types.tspackages/ai-client/tests/chat-client-queue.test.tspackages/ai-preact/src/index.tspackages/ai-preact/src/types.tspackages/ai-preact/src/use-chat.tspackages/ai-react/src/index.tspackages/ai-react/src/types.tspackages/ai-react/src/use-chat.tspackages/ai-solid/src/index.tspackages/ai-solid/src/types.tspackages/ai-solid/src/use-chat.tspackages/ai-svelte/src/create-chat.svelte.tspackages/ai-svelte/src/index.tspackages/ai-svelte/src/types.tspackages/ai-vue/src/index.tspackages/ai-vue/src/types.tspackages/ai-vue/src/use-chat.tspackages/ai/skills/ai-core/chat-experience/SKILL.md
🚧 Files skipped from review as they are similar to previous changes (22)
- packages/ai-svelte/src/index.ts
- packages/ai-client/src/index.ts
- packages/ai-preact/src/index.ts
- packages/ai-angular/src/index.ts
- packages/ai-solid/src/index.ts
- packages/ai-react/src/index.ts
- packages/ai-solid/src/types.ts
- packages/ai-vue/src/index.ts
- docs/chat/streaming.md
- packages/ai-vue/src/types.ts
- packages/ai-preact/src/types.ts
- packages/ai-angular/src/types.ts
- packages/ai-preact/src/use-chat.ts
- packages/ai-react/src/use-chat.ts
- packages/ai-react/src/types.ts
- packages/ai-svelte/src/types.ts
- packages/ai-svelte/src/create-chat.svelte.ts
- packages/ai-solid/src/use-chat.ts
- packages/ai-client/src/types.ts
- packages/ai-vue/src/use-chat.ts
- packages/ai-angular/src/inject-chat.ts
- packages/ai-client/src/chat-client.ts
Comment on lines
+786
to
+842
| it('interrupt during FIFO drain keeps remaining items for after the interrupt', async () => { | ||
| const deferred1 = createDeferred<void>() | ||
| const deferred2 = createDeferred<void>() | ||
| let call = 0 | ||
| const connection: ConnectConnectionAdapter = { | ||
| async *connect() { | ||
| call += 1 | ||
| if (call === 1) { | ||
| await deferred1.promise | ||
| } else if (call === 2) { | ||
| await deferred2.promise | ||
| } | ||
| yield* createTextChunks('done', `msg-${call}`) | ||
| }, | ||
| } | ||
| const client = new ChatClient({ connection }) | ||
| const firstSend = client.sendMessage('first') | ||
| await vi.waitFor(() => { | ||
| expect(client.getIsLoading()).toBe(true) | ||
| }) | ||
| await client.sendMessage('second') | ||
| await client.sendMessage('third') | ||
| deferred1.resolve() | ||
| // Wait until FIFO drain has started streaming "second". | ||
| await vi.waitFor(() => { | ||
| const users = client.getMessages().filter((m) => m.role === 'user') | ||
| expect(users.map((m) => m.parts[0])).toEqual([ | ||
| { type: 'text', content: 'first' }, | ||
| { type: 'text', content: 'second' }, | ||
| ]) | ||
| expect(client.getIsLoading()).toBe(true) | ||
| }) | ||
| // "third" should still be queued while "second" streams. | ||
| expect(client.getQueue().map((m) => m.content)).toEqual(['third']) | ||
| const interruptSend = client.sendMessage('urgent', undefined, { | ||
| whenBusy: 'interrupt', | ||
| }) | ||
| // Interrupt does not flush remaining queue. | ||
| expect(client.getQueue().map((m) => m.content)).toEqual(['third']) | ||
| deferred2.resolve() | ||
| await Promise.all([firstSend, interruptSend]) | ||
| await vi.waitFor(() => { | ||
| expect(client.getQueue()).toEqual([]) | ||
| }) | ||
| const users = client.getMessages().filter((m) => m.role === 'user') | ||
| expect(users.map((m) => m.parts[0])).toEqual([ | ||
| { type: 'text', content: 'first' }, | ||
| { type: 'text', content: 'second' }, | ||
| { type: 'text', content: 'urgent' }, | ||
| { type: 'text', content: 'third' }, | ||
| ]) | ||
| }) |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Assert that the active FIFO drain is actually aborted.
This verifies ordering and queue retention, but never checks that "second" receives cancellation.
Proposed assertion
const deferred2 = createDeferred<void>()
let call = 0
+ let secondSignal: AbortSignal | undefined
const connection: ConnectConnectionAdapter = {
- async *connect() {+ async *connect(_messages, _data, abortSignal) {
call += 1
if (call === 1) {
await deferred1.promise
} else if (call === 2) {
+ secondSignal = abortSignal
await deferred2.promise
}
@@
const interruptSend = client.sendMessage('urgent', undefined, {
whenBusy: 'interrupt',
})
+ await vi.waitFor(() => {+ expect(secondSignal?.aborted).toBe(true)+ })📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| it('interrupt during FIFO drain keeps remaining items for after the interrupt',async()=>{ | |
| constdeferred1=createDeferred<void>() | |
| constdeferred2=createDeferred<void>() | |
| letcall=0 | |
| constconnection: ConnectConnectionAdapter={ | |
| async*connect(){ | |
| call+=1 | |
| if(call===1){ | |
| awaitdeferred1.promise | |
| }elseif(call===2){ | |
| awaitdeferred2.promise | |
| } | |
| yield*createTextChunks('done',`msg-${call}`) | |
| }, | |
| } | |
| constclient=newChatClient({ connection }) | |
| constfirstSend=client.sendMessage('first') | |
| awaitvi.waitFor(()=>{ | |
| expect(client.getIsLoading()).toBe(true) | |
| }) | |
| awaitclient.sendMessage('second') | |
| awaitclient.sendMessage('third') | |
| deferred1.resolve() | |
| // Wait until FIFO drain has started streaming "second". | |
| awaitvi.waitFor(()=>{ | |
| constusers=client.getMessages().filter((m)=>m.role==='user') | |
| expect(users.map((m)=>m.parts[0])).toEqual([ | |
| {type: 'text',content: 'first'}, | |
| {type: 'text',content: 'second'}, | |
| ]) | |
| expect(client.getIsLoading()).toBe(true) | |
| }) | |
| // "third" should still be queued while "second" streams. | |
| expect(client.getQueue().map((m)=>m.content)).toEqual(['third']) | |
| constinterruptSend=client.sendMessage('urgent',undefined,{ | |
| whenBusy: 'interrupt', | |
| }) | |
| // Interrupt does not flush remaining queue. | |
| expect(client.getQueue().map((m)=>m.content)).toEqual(['third']) | |
| deferred2.resolve() | |
| awaitPromise.all([firstSend,interruptSend]) | |
| awaitvi.waitFor(()=>{ | |
| expect(client.getQueue()).toEqual([]) | |
| }) | |
| constusers=client.getMessages().filter((m)=>m.role==='user') | |
| expect(users.map((m)=>m.parts[0])).toEqual([ | |
| {type: 'text',content: 'first'}, | |
| {type: 'text',content: 'second'}, | |
| {type: 'text',content: 'urgent'}, | |
| {type: 'text',content: 'third'}, | |
| ]) | |
| }) | |
| it('interrupt during FIFO drain keeps remaining items for after the interrupt',async()=>{ | |
| constdeferred1=createDeferred<void>() | |
| constdeferred2=createDeferred<void>() | |
| letcall=0 | |
| letsecondSignal: AbortSignal|undefined | |
| constconnection: ConnectConnectionAdapter={ | |
| async*connect(_messages,_data,abortSignal){ | |
| call+=1 | |
| if(call===1){ | |
| awaitdeferred1.promise | |
| }elseif(call===2){ | |
| secondSignal=abortSignal | |
| awaitdeferred2.promise | |
| } | |
| yield*createTextChunks('done',`msg-${call}`) | |
| }, | |
| } | |
| constclient=newChatClient({ connection }) | |
| constfirstSend=client.sendMessage('first') | |
| awaitvi.waitFor(()=>{ | |
| expect(client.getIsLoading()).toBe(true) | |
| }) | |
| awaitclient.sendMessage('second') | |
| awaitclient.sendMessage('third') | |
| deferred1.resolve() | |
| // Wait until FIFO drain has started streaming "second". | |
| awaitvi.waitFor(()=>{ | |
| constusers=client.getMessages().filter((m)=>m.role==='user') | |
| expect(users.map((m)=>m.parts[0])).toEqual([ | |
| {type: 'text',content: 'first'}, | |
| {type: 'text',content: 'second'}, | |
| ]) | |
| expect(client.getIsLoading()).toBe(true) | |
| }) | |
| // "third" should still be queued while "second" streams. | |
| expect(client.getQueue().map((m)=>m.content)).toEqual(['third']) | |
| constinterruptSend=client.sendMessage('urgent',undefined,{ | |
| whenBusy: 'interrupt', | |
| }) | |
| awaitvi.waitFor(()=>{ | |
| expect(secondSignal?.aborted).toBe(true) | |
| }) | |
| // Interrupt does not flush remaining queue. | |
| expect(client.getQueue().map((m)=>m.content)).toEqual(['third']) | |
| deferred2.resolve() | |
| awaitPromise.all([firstSend,interruptSend]) | |
| awaitvi.waitFor(()=>{ | |
| expect(client.getQueue()).toEqual([]) | |
| }) | |
| constusers=client.getMessages().filter((m)=>m.role==='user') | |
| expect(users.map((m)=>m.parts[0])).toEqual([ | |
| {type: 'text',content: 'first'}, | |
| {type: 'text',content: 'second'}, | |
| {type: 'text',content: 'urgent'}, | |
| {type: 'text',content: 'third'}, | |
| ]) | |
| }) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-client/tests/chat-client-queue.test.ts` around lines 786 - 842,
Update the “interrupt during FIFO drain” test around the second connection
attempt to track its cancellation, such as by observing the async connection’s
abort signal or cancellation callback. After sending “urgent” with whenBusy:
'interrupt', assert that the active “second” request was aborted while
preserving the existing ordering and queued “third” assertions.
Comment on lines
+445
to
+448
| **Drain vs flush:** queued messages auto-send only after a **successful** | ||
| settle. They are **discarded** on stream error/abort of the active | ||
| generation, `stop()`, `clear()`, `unsubscribe()`, and `reload()`. | ||
| `interrupt` does not flush. |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Clarify the abort exception for interrupts.
“Abort of the active generation” contradicts “interrupt does not flush,” because interrupting aborts the current stream while preserving queued items. Explicitly distinguish interrupt-triggered aborts from aborts that discard the queue.
🧰 Tools
🪛 SkillSpector (2.3.11)
[warning] 372: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.
Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.
(Memory Poisoning (MP2))
[warning] 372: [MP2] Context Window Stuffing: Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.
Remediation: Implement context-window management that detects and rejects padding or stuffing attempts. Prioritize system instructions over user-injected content.
(Memory Poisoning (MP2))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai/skills/ai-core/chat-experience/SKILL.md` around lines 445 - 448,
Clarify the “Drain vs flush” documentation to distinguish interrupt-triggered
stream aborts from other active-generation aborts: preserve queued messages
across `interrupt`, while discarding them for stream errors, non-interrupt
aborts, `stop()`, `clear()`, `unsubscribe()`, and `reload()`. Retain the
existing rule that only a successful settle auto-sends queued messages.
tombeckenham
self-requested a review
July 17, 2026 07:20
tombeckenham
approved these changes
Jul 17, 2026
tombeckenham
left a comment
Contributor
There was a problem hiding this comment.
Fixed a couple of things with the fifo queue. It now works nicely.
Uh oh!
There was an error while loading. Please reload this page.
Merged
tombeckenham pushed a commit
that referenced
this pull request
Aug 17, 2026
Adds `@tanstack/ai-octane`, an Octane binding for TanStack AI. This is a port of `@octanejs/tanstack-ai@0.0.11`, which lived in the octanejs/octane repo as a temporary stopgap; the code moves here essentially unchanged apart from the rename. Covers the `@tanstack/ai-react` hook surface (useChat, useRealtimeChat, useMcpAppBridge, useGeneration, useGenerateImage/Audio/Speech/Video, useTranscription, useSummarize, useAudioRecorder) plus the 30 `@tanstack/ai-client` convenience re-exports, reusing `@tanstack/ai` and `@tanstack/ai-client` unchanged. SSR through `octane/server` is tested. Packaging: like Svelte packages shipping `.svelte`, this one publishes uncompiled source. The hook modules are `.tsrx`, compiled by the consumer's Octane plugin, so there is no `dist`/`build` target and no publint `test:build`; `octane` is a required peer. The `.tsrx.d.ts` companions are checked declaration emits, so `tsc` still verifies the full public generic surface. Deviations from a byte-for-byte move, all recorded in status.json: - Conformance runs on happy-dom rather than jsdom. This repo pins jsdom ^27, whose `Blob` has no `arrayBuffer()` (Octane pins ^29, which does) and the useAudioRecorder cases need it. Bumping jsdom for one package would break sherif's cross-package version consistency. - Two useChat cases asserted that a second concurrent sendMessage is dropped. #900 made client-side queueing the default, so both now assert queue-and-deliver-in-order, mirroring the current ai-react tests. The hook needed no change — queueing lives in ChatClient. - The differential parity test stays in the Octane repo: it byte-compares streamed output against real @tanstack/ai-react, which would make a sibling workspace package a pinned test dependency. - Unused vendored test helpers were dropped rather than carried as dead code (knip). - The generated .tsrx.d.ts companions are excluded from lint; a regeneration would undo any hand-fix. Known gap: the port is baselined against @tanstack/ai-react 0.17.0 while this repo is at 0.18.1. The interrupts overhaul (#970) and server persistence / browser-refresh durability (#984) are not yet reflected in the Octane hooks, and `typetests/` is therefore not wired into CI — its 4 errors are confined to one tool-input-inference case that fails identically against @tanstack/ai-react, so it is not a port defect. status.json lists the exact type-surface delta. Parity catch-up is follow-up work. pnpm-workspace.yaml excludes octane@0.1.17 and @octanejs/testing-library@0.1.14 from minimumReleaseAge: Octane publishes the pair in lockstep (testing-library pins an exact octane peer) and releases roughly daily, so the newest pair is essentially always inside the 24h window. 143 tests pass (142 conformance + 1 SSR), no skips or todos. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tombeckenham pushed a commit
that referenced
this pull request
Aug 20, 2026
Adds `@tanstack/ai-octane`, an Octane binding for TanStack AI. This is a port of `@octanejs/tanstack-ai@0.0.11`, which lived in the octanejs/octane repo as a temporary stopgap; the code moves here essentially unchanged apart from the rename. Covers the `@tanstack/ai-react` hook surface (useChat, useRealtimeChat, useMcpAppBridge, useGeneration, useGenerateImage/Audio/Speech/Video, useTranscription, useSummarize, useAudioRecorder) plus the 30 `@tanstack/ai-client` convenience re-exports, reusing `@tanstack/ai` and `@tanstack/ai-client` unchanged. SSR through `octane/server` is tested. Packaging: like Svelte packages shipping `.svelte`, this one publishes uncompiled source. The hook modules are `.tsrx`, compiled by the consumer's Octane plugin, so there is no `dist`/`build` target and no publint `test:build`; `octane` is a required peer. The `.tsrx.d.ts` companions are checked declaration emits, so `tsc` still verifies the full public generic surface. Deviations from a byte-for-byte move, all recorded in status.json: - Conformance runs on happy-dom rather than jsdom. This repo pins jsdom ^27, whose `Blob` has no `arrayBuffer()` (Octane pins ^29, which does) and the useAudioRecorder cases need it. Bumping jsdom for one package would break sherif's cross-package version consistency. - Two useChat cases asserted that a second concurrent sendMessage is dropped. #900 made client-side queueing the default, so both now assert queue-and-deliver-in-order, mirroring the current ai-react tests. The hook needed no change — queueing lives in ChatClient. - The differential parity test stays in the Octane repo: it byte-compares streamed output against real @tanstack/ai-react, which would make a sibling workspace package a pinned test dependency. - Unused vendored test helpers were dropped rather than carried as dead code (knip). - The generated .tsrx.d.ts companions are excluded from lint; a regeneration would undo any hand-fix. Known gap: the port is baselined against @tanstack/ai-react 0.17.0 while this repo is at 0.18.1. The interrupts overhaul (#970) and server persistence / browser-refresh durability (#984) are not yet reflected in the Octane hooks, and `typetests/` is therefore not wired into CI — its 4 errors are confined to one tool-input-inference case that fails identically against @tanstack/ai-react, so it is not a port defect. status.json lists the exact type-surface delta. Parity catch-up is follow-up work. pnpm-workspace.yaml excludes octane@0.1.17 and @octanejs/testing-library@0.1.14 from minimumReleaseAge: Octane publishes the pair in lockstep (testing-library pins an exact octane peer) and releases roughly daily, so the newest pair is essentially always inside the 24h window. 143 tests pass (142 conformance + 1 SSR), no skips or todos. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tombeckenham pushed a commit
that referenced
this pull request
Aug 20, 2026
Adds `@tanstack/ai-octane`, an Octane binding for TanStack AI. This is a port of `@octanejs/tanstack-ai@0.0.11`, which lived in the octanejs/octane repo as a temporary stopgap; the code moves here essentially unchanged apart from the rename. Covers the `@tanstack/ai-react` hook surface (useChat, useRealtimeChat, useMcpAppBridge, useGeneration, useGenerateImage/Audio/Speech/Video, useTranscription, useSummarize, useAudioRecorder) plus the 30 `@tanstack/ai-client` convenience re-exports, reusing `@tanstack/ai` and `@tanstack/ai-client` unchanged. SSR through `octane/server` is tested. Packaging: like Svelte packages shipping `.svelte`, this one publishes uncompiled source. The hook modules are `.tsrx`, compiled by the consumer's Octane plugin, so there is no `dist`/`build` target and no publint `test:build`; `octane` is a required peer. The `.tsrx.d.ts` companions are checked declaration emits, so `tsc` still verifies the full public generic surface. Deviations from a byte-for-byte move, all recorded in status.json: - Conformance runs on happy-dom rather than jsdom. This repo pins jsdom ^27, whose `Blob` has no `arrayBuffer()` (Octane pins ^29, which does) and the useAudioRecorder cases need it. Bumping jsdom for one package would break sherif's cross-package version consistency. - Two useChat cases asserted that a second concurrent sendMessage is dropped. #900 made client-side queueing the default, so both now assert queue-and-deliver-in-order, mirroring the current ai-react tests. The hook needed no change — queueing lives in ChatClient. - The differential parity test stays in the Octane repo: it byte-compares streamed output against real @tanstack/ai-react, which would make a sibling workspace package a pinned test dependency. - Unused vendored test helpers were dropped rather than carried as dead code (knip). - The generated .tsrx.d.ts companions are excluded from lint; a regeneration would undo any hand-fix. Known gap: the port is baselined against @tanstack/ai-react 0.17.0 while this repo is at 0.18.1. The interrupts overhaul (#970) and server persistence / browser-refresh durability (#984) are not yet reflected in the Octane hooks, and `typetests/` is therefore not wired into CI — its 4 errors are confined to one tool-input-inference case that fails identically against @tanstack/ai-react, so it is not a port defect. status.json lists the exact type-surface delta. Parity catch-up is follow-up work. pnpm-workspace.yaml excludes octane@0.1.17 and @octanejs/testing-library@0.1.14 from minimumReleaseAge: Octane publishes the pair in lockstep (testing-library pins an exact octane peer) and releases roughly daily, so the newest pair is essentially always inside the 24h window. 143 tests pass (142 conformance + 1 SSR), no skips or todos. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Today
sendMessagesilently drops any message sent while a stream is in flight. This adds a first-class, observable, configurable message queue: messages sent mid-stream are queued by default, rendered separately, auto-sent when the stream settles, and cancellable before they go out.Behavior
whenBusy—queue(hold + auto-send),drop(ignore, the old behavior),interrupt(abort current stream, send now).drain—fifo(one at a time, in order) orbatch(merge all queued into one send: strings joined by\n, multimodal content concatenated).maxSize/onOverflow— cap the queue;rejectthe incoming send or evict the oldest.WhenBusystring shorthand or aQueueStrategyfunction for full control.stop()/clear()/unsubscribe()and on stream error (never strands or mis-orders queued sends).messages, never sent to the wire until drained.Sends while streaming are now queued by default (previously dropped). Opt back into the old behavior with
queue: 'drop'. Flagged as a minor bump in the changeset.Surface
@tanstack/ai-client— core:queueoption,QueuedMessage,getQueue(),cancelQueued(),onQueueChange, FIFO/batch drain, lifecycle + error flush, devtools snapshot.@tanstack/ai-react/-solid/-vue/-svelte/-preact— exposequeue+cancelQueued, per-send{ whenBusy }override;ai-vue-uiforwards automatically.Tests & docs
ai-client(eachwhenBusymode, FIFO order, batch merge incl. multimodal,maxSize/overflow, cancel, flush-on-stop/clear/error) + framework hook tests updated for queue-by-default.testing/e2e/tests/queue.spec.ts): send → queue-while-streaming → cancel one → assert only the uncancelled message is delivered, in order.docs/chat/streaming.md;chat-experienceagent skill updated.pnpm test:prgreen across all 50 projects.🤖 Generated with Claude Code
Summary by CodeRabbit
sendMessagewhile a stream is active, with configurablewhenBusy(queue/drop/interrupt) and drain/overflow strategies.queuestate,cancelQueued(id), and per-send{ whenBusy }overrides across chat hooks/components.