Uh oh!
There was an error while loading. Please reload this page.
Support client-side tools in DurableAgent - #1329
Merged
Merged
Conversation
Tools without an `execute` function now pause the agent loop and return `clientToolCalls` in the result, enabling human-in-the-loop and client-side tool execution patterns. Fixes#847. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 4b4fb1a The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Stream Benchmarks(includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (42 failed)astro (4 failed):
example (5 failed):
express (5 failed):
fastify (3 failed):
hono (4 failed):
nextjs-turbopack (6 failed):
nextjs-webpack (5 failed):
nitro (2 failed):
nuxt (4 failed):
sveltekit (3 failed):
vite (1 failed):
🐘 Local Postgres (1 failed)sveltekit-stable (1 failed):
🌍 Community Worlds (56 failed)mongodb (3 failed):
redis (2 failed):
turso (51 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
❌ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
Contributor
Uh oh!
There was an error while loading. Please reload this page.
Contributor
There was a problem hiding this comment.
Pull request overview
Updates DurableAgent to support AI SDK-style “client-side” tools (tools without execute) by pausing the agent loop and returning unresolved tool calls to the caller, rather than throwing.
Changes:
- Add
ClientToolCalltype andclientToolCallsfield toDurableAgentStreamResult. - Pause the loop when non-provider tool calls lack an
executefunction, while still executing any executable/provider tools in the same step. - Extract
resolveProviderToolResulthelper and add new test coverage for client-side tool behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| packages/ai/src/agent/durable-agent.ts | Implements pause-and-return behavior for tools without execute, adds result typing, and refactors provider tool result resolution. |
| packages/ai/src/agent/durable-agent.test.ts | Adds tests covering pause behavior, mixed tool steps, and onFinish behavior when pausing. |
| .changeset/client-side-tools.md | Declares a patch release for the new client-side tools behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| toolCallId: 'ask-user-call-id', | ||
| toolName: 'askUser', | ||
| input: { question: 'What is your name?' }, | ||
| }); |
There was a problem hiding this comment.
The new client-side tool tests validate clientToolCalls, but they don't assert that result.messages contains the assistant tool-call message needed to resume (or that server/provider tool results are preserved when pausing). Adding assertions around the returned messages shape/content for the pause cases would catch regressions where the agent returns only the original input messages.
Suggested change
| }); | |
| }); | |
| // Result messages should match the messages returned by the iterator | |
| expect(result.messages).toBeDefined(); | |
| expect(result.messages).toEqual(mockMessages); |
Comment on lines
+899
to
+906
| const messages = (finalMessages ?? | ||
| options.messages) as unknown as ModelMessage[]; | ||
| if (options.onFinish && !wasAborted) { | ||
| await options.onFinish({ | ||
| steps, | ||
| messages: messages as ModelMessage[], | ||
| experimental_context: experimentalContext, |
There was a problem hiding this comment.
In the client-side tool stop branch, messages is derived from (finalMessages ?? options.messages), but finalMessages is only set on normal iterator completion. When stopping early for client-side tools, this returns the original input messages and drops the assistant tool-call message from iterMessages, making it impossible for callers to resume by appending tool results to the correct tool calls. Return iterMessages (or a derived message list that includes the tool-call assistant message) instead of falling back to options.messages here, and ensure onFinish receives the same message list.
Comment on lines
+834
to
+841
| const executableToolCalls = nonProviderToolCalls.filter((tc) => { | ||
| const tool = (effectiveTools as ToolSet)[tc.toolName]; | ||
| return tool && typeof tool.execute === 'function'; | ||
| }); | ||
| const clientSideToolCalls = nonProviderToolCalls.filter((tc) => { | ||
| const tool = (effectiveTools as ToolSet)[tc.toolName]; | ||
| return !tool || typeof tool.execute !== 'function'; | ||
| }); |
There was a problem hiding this comment.
clientSideToolCalls currently includes tool calls where the tool is missing (!tool). That changes prior behavior (missing tools threw) and will cause the agent to pause and return an unresolved call for an unknown tool name (likely a model hallucination) instead of failing fast. Consider treating !tool as an error and only pausing for tools that exist but lack an execute function.
Comment on lines
+873
to
+877
| await writer.write({ | ||
| type: 'tool-output-available' as const, | ||
| toolCallId: result.toolCallId, | ||
| output: result.output.value, | ||
| } as UIMessageChunk); |
There was a problem hiding this comment.
When pausing for client-side tools, resolved executable/provider tool outputs are written directly to options.writable, but those chunks are not appended to allUIChunks. If collectUIMessages is enabled, the returned uiMessages will be missing these tool-output chunks (since uiChunks yielded by the iterator don't include tool outputs until after iterator.next(toolResults)). Add the written tool-output-available chunks to allUIChunks when collectUIChunks is true (or reuse the existing writeToolOutputToUI helper logic) so uiMessages reflects what was actually streamed.
Suggested change
| awaitwriter.write({ | |
| type: 'tool-output-available'asconst, | |
| toolCallId: result.toolCallId, | |
| output: result.output.value, | |
| }asUIMessageChunk); | |
| constchunk: UIMessageChunk={ | |
| type: 'tool-output-available'asconst, | |
| toolCallId: result.toolCallId, | |
| output: result.output.value, | |
| }; | |
| awaitwriter.write(chunk); | |
| if(collectUIChunks){ | |
| allUIChunks.push(chunk); | |
| } |
Comment on lines
+846
to
+921
| // Execute any executable tools that were also called in this step | ||
| const executableResults = await Promise.all( | ||
| executableToolCalls.map( | ||
| (toolCall): Promise<LanguageModelV2ToolResultPart> => | ||
| executeTool( | ||
| toolCall, | ||
| effectiveTools as ToolSet, | ||
| iterMessages, | ||
| experimentalContext, | ||
| options.experimental_repairToolCall as ToolCallRepairFunction<ToolSet> | ||
| ) | ||
| ) | ||
| ); | ||
| // Collect provider tool results | ||
| const providerResults: LanguageModelV2ToolResultPart[] = | ||
| providerToolCalls.map((toolCall) => | ||
| resolveProviderToolResult(toolCall, providerExecutedToolResults) | ||
| ); | ||
| // Combine executable and provider results, then write to UI | ||
| const resolvedResults = [...executableResults, ...providerResults]; | ||
| if (resolvedResults.length > 0) { | ||
| // Write resolved tool results to the UI stream | ||
| const writer = options.writable.getWriter(); | ||
| try { | ||
| for (const result of resolvedResults) { | ||
| await writer.write({ | ||
| type: 'tool-output-available' as const, | ||
| toolCallId: result.toolCallId, | ||
| output: result.output.value, | ||
| } as UIMessageChunk); | ||
| } | ||
| } finally { | ||
| writer.releaseLock(); | ||
| } | ||
| } | ||
| // Parse client-side tool call inputs | ||
| const parsedClientToolCalls: ClientToolCall[] = | ||
| clientSideToolCalls.map((tc) => ({ | ||
| toolCallId: tc.toolCallId, | ||
| toolName: tc.toolName, | ||
| input: JSON.parse(tc.input || '{}'), | ||
| })); | ||
| // Close the stream and call onFinish before returning | ||
| const sendFinish = options.sendFinish ?? true; | ||
| const preventClose = options.preventClose ?? false; | ||
| if (sendFinish || !preventClose) { | ||
| await closeStream(options.writable, preventClose, sendFinish); | ||
| } | ||
| const messages = (finalMessages ?? | ||
| options.messages) as unknown as ModelMessage[]; | ||
| if (options.onFinish && !wasAborted) { | ||
| await options.onFinish({ | ||
| steps, | ||
| messages: messages as ModelMessage[], | ||
| experimental_context: experimentalContext, | ||
| experimental_output: undefined as OUTPUT, | ||
| }); | ||
| } | ||
| const uiMessages = collectUIChunks | ||
| ? await convertChunksToUIMessages(allUIChunks) | ||
| : undefined; | ||
| return { | ||
| messages: messages as ModelMessage[], | ||
| steps, | ||
| experimental_output: undefined as OUTPUT, | ||
| uiMessages, | ||
| clientToolCalls: parsedClientToolCalls, | ||
| }; |
There was a problem hiding this comment.
In the pause path, executable/provider tools are executed/resolved, but their results are neither appended to the returned messages nor exposed in the return value. This makes resuming hard/impossible (especially for provider-executed tool results, which the caller may not be able to reconstruct). Consider returning these resolved tool results (or returning an updated messages array that includes a role: 'tool' message with the resolved results) so the caller can resume without re-executing server tools and without losing provider results.
Use `toolCalls` and `toolResults` on DurableAgentStreamResult to match the AI SDK's GenerateTextResult convention. Consumers find unresolved client-side tool calls by diffing toolCalls vs toolResults — the same pattern used with useChat/addToolOutput. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Push directly to iterMessages (LanguageModelV2Prompt) instead of double-casting through ModelMessage[]. Single cast at the return point matches the existing pattern at the end of stream(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
gr2m
approved these changes
Mar 11, 2026
| > | ||
| ): LanguageModelV2ToolResultPart { | ||
| const streamResult = providerExecutedToolResults?.get(toolCall.toolCallId); | ||
| if (streamResult) { |
Contributor
There was a problem hiding this comment.
nit: I would reverse the check and do an early exit to have less code indentation.
But not sure if agents care 🤣
ContributorAuthor
There was a problem hiding this comment.
Done in 4b4fb1a — early exit with the warning/empty result, main path unindented.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
executefunction now pause the agent loop instead of throwingTool X does not have an execute functiontoolCallsandtoolResultsmatching the AI SDK'sGenerateTextResultconventionmessagesinclude the full conversation (assistant tool-call message + resolved tool results) so callers can resume seamlesslytool-input-availablechunks for all tool calls viadoStreamStep, souseChatsees client-side tool calls without any frontend changesUsage
Defining a client-side tool
Define a tool without an
executefunction — just like in the AI SDK:Server-side: detecting unresolved tool calls
When the model calls a client-side tool, the agent loop pauses and returns. Check
toolCallsvstoolResultsto find unresolved calls — the same pattern as the AI SDK:Frontend: using with
useChatOn the frontend, client-side tool calls appear automatically as message parts via
useChat. UseaddToolOutputto provide results:Motivation
Fixes#847. The AI SDK allows tools without
executefunctions for human-in-the-loop, client-side tool execution, and other patterns where tool results come from outside the server. DurableAgent previously threw an error for these tools; now it matches the AI SDK behavior by pausing the loop and exposing unresolved calls on the result.Test plan
executestops the loop,toolCallshas entry,toolResultsis emptytoolCallshas both,toolResultsonly has executedonFinishcallback fires when stopping for client-side toolstoolCalls/toolResultsmessagesreferenceiterMessages(includes assistant tool-call message)@workflow/aitest suite passes (84 tests)🤖 Generated with Claude Code