Uh oh!
There was an error while loading. Please reload this page.
fix(ai): preserve provider tool identity across step boundaries - #1663
Merged
Conversation
Port of vercel/ai#14229. Provider tools (e.g. anthropic.tools.webSearch) were converted to plain function tools in toolsToModelTools, stripping type, id, and args fields. This caused providers like Anthropic Gateway to not recognize them as provider-executed tools. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 9bc8db3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (64 failed)mongodb (3 failed):
redis (3 failed):
turso (58 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Stream Benchmarks(includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
TooTallNate
approved these changes
Apr 8, 2026
TooTallNate
left a comment
Member
There was a problem hiding this comment.
Port of vercel/ai#14229. Clean fix with solid test coverage.
What I verified
Core fix (toolsToModelTools): The type === 'provider' check correctly short-circuits before the function tool serialization path. Provider tools get { type: 'provider', id, name, args } — preserving identity. Function tools are unchanged. The return type widens from LanguageModelV3FunctionTool[] to Array<LanguageModelV3FunctionTool | LanguageModelV3ProviderTool>, which is compatible with the downstream doStreamStep parameter type (LanguageModelV3CallOptions['tools'] — already a union).
args default:(tool as any).args ?? {} defaults to empty object when args is undefined. This matches the AI SDK convention where provider tools may omit args (e.g. anthropic.tools.codeExecution). The test covers this case.
Mock model: The provider-tool-call variant correctly emits both tool-call with providerExecuted: true AND tool-result in the same stream — matching how real providers (Anthropic Gateway) return provider-executed tool results.
E2E workflows:agentProviderToolE2e (provider-only) and agentMixedToolsE2e (provider + function) cover the two critical scenarios. Both use as any for the provider tool definition, which is fine since ToolSet doesn't natively type provider tools.
Changeset:patch for @workflow/ai — correct. The changeset description is clear and references the root cause.
One observation (non-blocking)
The (tool as any).type === 'provider' cast works but is opaque. If ai ever exports a ProviderTool type or type guard (e.g. isProviderTool(tool)), consider switching to that for type safety. For now, as any matches the upstream fix pattern.
LGTM.
| tools: ToolSet | ||
| ): Promise<LanguageModelV3FunctionTool[]> { | ||
| ): Promise<Array<LanguageModelV3FunctionTool | LanguageModelV3ProviderTool>> { | ||
| return Promise.all( |
Member
There was a problem hiding this comment.
Non-blocking:(tool as any).type === 'provider' is the same pattern used in the upstream AI SDK fix (vercel/ai#14229). If ai later exports a type guard or the ToolSet type is widened to include provider tools natively, this could be made type-safe. Fine for now.
…github.com> I, Gregor Martynus <39992+gr2m@users.noreply.github.com>, hereby add my Signed-off-by to this commit: b1930b3 Signed-off-by: Gregor Martynus <39992+gr2m@users.noreply.github.com>
gr2m
enabled auto-merge (squash)
April 9, 2026 01:01
Uh oh!
There was an error while loading. Please reload this page.
ghost
commented
Apr 9, 2026
ghost
pushed a commit
that referenced
this pull request
Apr 9, 2026
![workflow-devkit-release-bot[bot]](https://github.laxmiang.work.gd/r/github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
* fix(ai): preserve provider tool identity across step boundaries Port of vercel/ai#14229. Provider tools (e.g. anthropic.tools.webSearch) were converted to plain function tools in toolsToModelTools, stripping type, id, and args fields. This caused providers like Anthropic Gateway to not recognize them as provider-executed tools. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * DCO Remediation Commit for Gregor Martynus <39992+gr2m@users.noreply.github.com> I, Gregor Martynus <39992+gr2m@users.noreply.github.com>, hereby add my Signed-off-by to this commit: b1930b3 Signed-off-by: Gregor Martynus <39992+gr2m@users.noreply.github.com> --------- Signed-off-by: Gregor Martynus <39992+gr2m@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
toolsToModelToolswas converting all tools intoLanguageModelV3FunctionToolobjects withtype: 'function', strippingtype: 'provider',id, andargsfrom provider tools (e.g.anthropic.tools.webSearch,webFetch,codeExecution)GatewayInternalServerError: Unexpected value(s) for the anthropic-beta headertool.type === 'provider'and returns aLanguageModelV3ProviderToolpreserving identity, while regular function tools continue to be serialized as beforeprovider-tool-callmock response type for testing provider-executed tool behavior end-to-endRelated issues & PRs
toolsToModelToolsto forwardstrict,inputExamples,providerOptionsfor function tools (complementary fix, different code path)Changes
packages/ai/src/agent/tools-to-model-tools.tstype: 'provider'and returnLanguageModelV3ProviderToolpreservingid,name,argspackages/ai/src/agent/tools-to-model-tools.test.tspackages/ai/src/providers/mock.tsprovider-tool-callvariant toMockResponseDescriptor(emitstool-callwithproviderExecuted: true+tool-result)workbench/example/workflows/100_durable_agent_e2e.tsagentProviderToolE2eandagentMixedToolsE2eworkflow functionspackages/core/e2e/e2e-agent.test.tsprovider toolsdescribe block with 2 E2E testsTest plan
pnpm --filter @workflow/ai test— 173 tests, 10 files)@workflow/aibuilds cleanly (pnpm --filter @workflow/ai build)cd workbench/example && pnpm build)agentProviderToolE2e— provider tool identity preserved across step boundariesagentMixedToolsE2e— mixed provider + function tools in same agent