Uh oh!
There was an error while loading. Please reload this page.
[ai] Reduce doStreamStep step boundary payload - #1932
Conversation
Move chunksToStep out of the step boundary, returning minimal raw aggregates (text, reasoning, files, sources, warnings, response metadata, finishReason, usage, providerMetadata) and reconstructing the full StepResult in streamTextIterator (workflow context). Also drops the separate `finish` field from the return — its data was fully duplicated by the StepResult fields the caller already needed. The user-facing StepResult exposed via `onStepFinish` and `steps[]` is unchanged. The event-log payload no longer carries: - StepResult's redundant tool-call lists (`dynamicToolCalls`, `staticToolCalls`, `staticToolResults`, `dynamicToolResults`) - The always-empty `toolResults` and `response.messages` arrays - The duplicate `content` and `reasoningText` aggregates - The dual base64 + Uint8Array file encoding (now expanded only after crossing the boundary) - `request.body` (a JSON dump of the input prompt the caller already has) and `finish` (its fields all appear on `step`) Closes#1929. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: e4066d4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Stream Benchmarks(includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
| warnings = chunk.warnings; | ||
| break; | ||
| case 'response-metadata': | ||
| responseMetadata = { |
There was a problem hiding this comment.
AI Review: This now replaces the entire response metadata object on every response-metadata chunk. Those fields are optional, and AI SDK stream handling treats metadata as cumulative, so a later partial chunk can drop an earlier id/timestamp before StepResult.response is rebuilt. Please merge non-null fields into the existing metadata instead of overwriting it.
shtefcs
commented
Jun 24, 2026
Production user data point + a couple of questions on merge readiness. We run V2 Decrypted
So for our workload the redundancy this PR removes is dominated by Two questions before we adopt:
Thanks for the work on this — happy to provide more traces if useful. |
A provider may emit multiple response-metadata chunks; a later partial chunk must not clobber an id/timestamp set by an earlier one. Mirror the AI SDK's `chunk.x ?? prev.x` merge so StepResult.response and the OTEL span attributes retain every populated field. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ce-stream-step-payload
VaguelySerious
left a comment
There was a problem hiding this comment.
AI review: no blocking issues
| case 'stream-start': | ||
| warnings = chunk.warnings; | ||
| break; | ||
| case 'response-metadata': |
There was a problem hiding this comment.
AI Review: Note
This is a real behavior change, not just a refactor: the old chunksToStep did chunks.find(c => c.type === 'response-metadata') (first-wins), whereas this now merges per-field across multiple response-metadata chunks (last-non-null-wins per field). It's the correct behavior (matches the AI SDK) and the new test covers it — but for providers that split metadata across chunks it can change the reconstructed response.id / response.modelId / response.timestamp and the derived model.provider. Worth a line in the changeset/PR description so it isn't a silent diff for anyone comparing step output across versions.
There was a problem hiding this comment.
(AI) Documented this behavior change in the PR description ("Behavior change worth noting"). Keeping the changeset to one terse line per repo convention.
| ...toolCalls.map(mapToolCall), | ||
| ], | ||
| text: raw.text, | ||
| reasoning: reasoning.map((r) => ({ |
There was a problem hiding this comment.
AI Review: Nit
reasoning is already materialized into its final { type: 'reasoning', text, providerOptions? } shape at the top of this function (the const reasoning = raw.reasoning.map(...) block). Re-mapping it here produces identical objects and the as SharedV3ProviderOptions cast is dead. This can collapse to just reasoning, (the earlier block already typed providerOptions correctly).
There was a problem hiding this comment.
(AI) Addressed in e4066d4 — collapsed to reuse the already-materialized reasoning array and dropped the dead cast.
| } else { | ||
| // Data is a base64 string. (URL is not currently supported here — | ||
| // matches prior behavior in chunksToStep.) | ||
| const binaryString = atob(data as string); |
There was a problem hiding this comment.
AI Review: Nit
RawFile.data is typed Uint8Array | string | URL, but this branch does atob(data as string) — a URL would be silently corrupted here. The behavior is carried over verbatim from the old chunksToStep (which only ever saw base64 strings), so it's not a regression, but the new explicit URL member in the type makes the gap visible. Either narrow RawFile.data to what we actually emit, or handle the URL/data-URL case (or at least guard + warn) so a provider that emits a URL-backed file doesn't produce a garbage uint8Array.
There was a problem hiding this comment.
(AI) Addressed in e4066d4 — RawFile.data is now derived from the V3 file stream part (string | Uint8Array), so a future widening (e.g. URL) breaks the base64-decode typecheck instead of silently corrupting bytes.
…ing remap - Derive RawFile.data from the V3 file stream part (string | Uint8Array) so a future widening (e.g. URL) breaks the base64-decode typecheck instead of silently corrupting bytes. - buildStepResult: reuse the already-materialized reasoning array instead of re-mapping it identically; drop now-unnecessary casts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
VaguelySerious
commented
Jun 26, 2026
(AI) Thanks for the production numbers — the On your two questions: 2. Metadata concern — this is addressed at the current head ( // packages/ai/src/agent/do-stream-step.tscase'response-metadata':
responseMetadata={id: chunk.id??responseMetadata?.id,modelId: chunk.modelId??responseMetadata?.modelId,timestamp: chunk.timestamp??responseMetadata?.timestamp,};break;It's covered by the regression test 1. Merge timeline — I'll leave the actual scheduling to Peter, but for status: the review comments are addressed, the changeset is in, and CI is green ( If you'd like to re-test, the PR's pre-release tarballs are hosted on the docs preview: {
"dependencies": {
"workflow": "https://workflow-docs-git-peter-issue-1929-reduce-stream-step-payload.vercel.sh/workflow.tgz",
"@workflow/ai": "https://workflow-docs-git-peter-issue-1929-reduce-stream-step-payload.vercel.sh/workflow-ai.tgz"
}
}Refreshed payload numbers from your isolated preview would be very welcome — especially confirmation that the |
karthikscale3
left a comment
There was a problem hiding this comment.
Checked StepResult reconstruction/public shape, response-metadata merging, raw file/reasoning aggregation, step-boundary payload/serialization concerns, changeset, and green CI.
Backport PR opened against |
Summary
chunksToStepout of thedoStreamStepstep boundary. The step now returns minimal raw aggregates (text,reasoning,files,sources,warnings,responseMetadata,rawFinishReason,usage,providerMetadata); the fullStepResultis reconstructed instreamTextIterator(workflow context) viabuildStepResult.finishfield from the return — every field it carried was already on the reconstructedStepResult.StepResultexposed viaonStepFinishandresult.steps[]is unchanged. The reduction is purely in the durable execution event log.What's no longer crossing the boundary:
StepResult's redundant tool-call lists (dynamicToolCalls,staticToolCalls,staticToolResults,dynamicToolResults)toolResultsandresponse.messagescontentandreasoningTextaggregatesbase64 + Uint8Arrayfile encoding (expanded only after the boundary)request.body(a JSON dump of the input prompt the caller already has)finish(itsfinishReason,usage,providerMetadataalready onstep)Internally, the V3
chunks[]array inside the step is also gone — fields are aggregated directly during the streaming transform.Behavior change worth noting
response-metadatais now merged per-field across chunks instead of first-wins. The oldchunksToStepdidchunks.find(c => c.type === 'response-metadata')(first chunk wins for all fields); reconstruction now merges each field with last-non-null-wins, matching the AI SDK. For providers that split metadata across multiple chunks this can change the reconstructedresponse.id/response.modelId/response.timestampand the derivedmodel.provider/model.modelId. This is a fix, not a regression, and is covered by a new test indo-stream-step.test.ts.Closes#1929.
Test plan
pnpm --filter @workflow/ai test— 187 passing, including updated mocks fordo-stream-step.test.tsandstream-text-iterator.test.tspnpm --filter @workflow/ai build— clean tscStepResultshape is identical)🤖 Generated with Claude Code
Ported to
@ai-sdk/workflow:vercel/ai#16434