Uh oh!
There was an error while loading. Please reload this page.
[core] Forward-port stream reconnect to getReadable level - #2318
Merged
Conversation
Adds transparent auto-reconnect for object/serialized streams at the getReadable level. On upstream error mid-run, the wrapper reopens the server stream at startIndex + consumedFrames by counting 4-byte length-prefixed wire frames. Clean EOF is treated as completion. Byte streams keep their existing raw, non-reconnecting path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
🦋 Changeset detectedLatest commit: 484ff35 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express Stream Benchmarks(includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
VaguelySerious
marked this pull request as ready for review
June 10, 2026 09:46
VaguelySerious
commented
Jun 10, 2026
MemberAuthor
(AI) Cross-PR context & merge order Together these make How the pieces fit:
Behavioural note: the reconnecting reader only reopens on a connection error — a clean close means "complete". So the client change is inert on its own and only takes effect once paired with the coordinated server-side change that ends a timed-out connection with an error rather than a silent close (handled separately). Shipping the client first is therefore safe and must precede that server change. Suggested order:
#1853 is independent and can land on its own schedule; it unblocks byte-stream reconnect as a future follow-up. |
This was referenced Jun 10, 2026
TooTallNate
approved these changes
Jun 10, 2026
TooTallNate
left a comment
Member
There was a problem hiding this comment.
Approve — faithful forward-port, correctly adapted to main's world API
I reviewed this against the stable PR (#1847) side by side, built @workflow/core from this branch, and ran the test suites locally: all 11 reconnecting-framed-stream tests plus the 199 serialization tests pass (210 total).
Port fidelity
The reconnect mechanism is identical to #1847 — frame counting, partial-frame discard with resume at startIndex + consumedFrames, clean-EOF-equals-completion, negative-startIndex opt-out, consecutive cap with reset-on-progress, absolute backstop. The adaptations are exactly the ones main's API requires and nothing more:
world.readFromStream(name, idx)→world.streams.get(runId, name, idx), withrunIdthreaded throughcreateReconnectingFramedStreamgetWorld()→await getWorldLazy()- Reviver hookup matches: byte streams keep
WorkflowServerReadableStream(no reconnect — no framing to count, explicitly deferred to the byte-framing follow-up), object streams get the reconnecting wrapper
The test file gained one main-only case (11 vs 10) and the mock world is adapted to the streams.get shape. Changeset is correctly scoped (@workflow/core: patch).
Notes
Same non-blocking design note as the stable PR: a transient failure of the reopen itself (
reconnect()→connect()throwing) is fatal rather than counted against the budget — onlyreader.read()errors are retried. Worth folding into the budgeted loop in a follow-up, here and on stable, so the wrapper is robust against the exact server blip it exists to survive.Heads-up on a pending textual conflict: #1853 (byte-stream framing) touches the same reviver region of
serialization.ts— specifically thevalue.type === 'bytes'branch this PR also edits (it adds framing dispatch where this PR adds the no-reconnect comment + scopedreadableconstruction). Whichever lands second gets a small but real merge conflict; the resolution is straightforward (byte branch gains framing dispatch, object branch keeps the reconnecting wrapper) but worth coordinating so the second rebase is deliberate rather than auto-resolved.CI: the three red
nextjs-webpackjobs (Local Postgres/Local Prod/Local Dev, stable lazyDiscovery disabled) are the known pre-existing failure onmain(Dynamic require of "stream"duringnext buildpage-data collection) — I verified the identical failure signature on main's latest Tests run. Not attributable to this PR. Everything else is green and the branch has no conflicts with current main.
Good to land. Same sequencing logic as the stable PR applies: this is safe to release ahead of the coordinated server-side behavior change, since the reconnect path simply never triggers until the server starts erroring timed-out connections instead of closing them.
This was referenced Jun 10, 2026
pranaygp added a commit
that referenced
this pull request
Jun 11, 2026
…-attributes-client * origin/main: (32 commits) feat(core): add optional namespace for queue topic prefix (#2305) [web-shared] Show precise durations in the new trace viewer (#2335) Validate unique workflow step IDs at build time (#2018) Move run attributes into their own detail card (#2327) [core] Forward-port stream reconnect to getReadable level (#2318) [docs] Add "Step executed multiple times" error page (#2310) Fix flickering on the detail panel when navigating the trace viewer (#2325) Version Packages (beta) (#2254) Patch vulnerable package dependencies (#2301) Deprecate DurableAgent and update it with WorkflowAgent in v5 docs (#2285) [codex] Fix partial world-local exclusive writes (#2296) [world-vercel] Validate ref resolve responses before use (#2035) fix: bump devalue to 5.8.1 (#2292) Handle lazy stream key request failures (#2257) fix(docs): declare Nitro auto-import globals for code samples (#2290) [core] Preserve event-log order in hook-vs-sleep replay races (#2171) (#2185) docs: add nitro changelog (#2232) fix(next): always apply turbopack content condition regardless of builder mode (#2253) Update @vercel/queue from 0.2.1 to 0.3.0 (#2255) [nest] Use AST-based CommonJS import rewriting (#2080) ... # Conflicts: # packages/world-local/src/storage/events-storage.ts
VaguelySerious added a commit
that referenced
this pull request
Jun 11, 2026
Addresses review feedback on #2318: a transient failure of the reopen itself (`connect()` throwing inside `reconnect()`) was fatal — only `reader.read()` errors were retried. A brief server unavailability during the reconnect window is exactly the blip this wrapper exists to survive, so count it against the reconnect budget and retry rather than erroring the stream. Budget exhaustion (a server that stays down) still terminates. Adds a test: a reopen that throws once then succeeds recovers transparently. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pranaygp added a commit
that referenced
this pull request
Jun 11, 2026
…hook-ready * origin-https/main: (119 commits) fix: settle aborted parallel steps before completing abortParallelWorkflow (#2244) Add native v4 workflow attribute events (#2226) Version Packages (beta) (#2326) [ci] Fix flaky windows unit tests (#2359) Capture Vercel runtime logs when e2e Vercel Prod lanes fail (#2356) Fix e2e failure reporting under vitest 4 and preserve fetch error causes (#2355) [core] Fix process crash from rejected waitUntil promises (#2336) [core] Remove duplicate `waitUntil` for suspension handler async operations (#2345) Prevent local tests from hanging (#2338) feat(core): add optional namespace for queue topic prefix (#2305) [web-shared] Show precise durations in the new trace viewer (#2335) Validate unique workflow step IDs at build time (#2018) Move run attributes into their own detail card (#2327) [core] Forward-port stream reconnect to getReadable level (#2318) [docs] Add "Step executed multiple times" error page (#2310) Fix flickering on the detail panel when navigating the trace viewer (#2325) Version Packages (beta) (#2254) Patch vulnerable package dependencies (#2301) Deprecate DurableAgent and update it with WorkflowAgent in v5 docs (#2285) [codex] Fix partial world-local exclusive writes (#2296) ... # Conflicts: # packages/core/src/runtime.ts # packages/core/src/runtime/suspension-handler.ts
pranaygp added a commit
that referenced
this pull request
Jun 11, 2026
…docs-run-idempotency * origin-https/main: (115 commits) [core] V2: unify wait+step queue dispatch in suspension processing (#1925) fix(world-local,world-postgres): make duplicate hook_created idempotent (#2295) docs(observability): remove MVP implementation detail bullet (#2367) fix: settle aborted parallel steps before completing abortParallelWorkflow (#2244) Add native v4 workflow attribute events (#2226) Version Packages (beta) (#2326) [ci] Fix flaky windows unit tests (#2359) Capture Vercel runtime logs when e2e Vercel Prod lanes fail (#2356) Fix e2e failure reporting under vitest 4 and preserve fetch error causes (#2355) [core] Fix process crash from rejected waitUntil promises (#2336) [core] Remove duplicate `waitUntil` for suspension handler async operations (#2345) Prevent local tests from hanging (#2338) feat(core): add optional namespace for queue topic prefix (#2305) [web-shared] Show precise durations in the new trace viewer (#2335) Validate unique workflow step IDs at build time (#2018) Move run attributes into their own detail card (#2327) [core] Forward-port stream reconnect to getReadable level (#2318) [docs] Add "Step executed multiple times" error page (#2310) Fix flickering on the detail panel when navigating the trace viewer (#2325) Version Packages (beta) (#2254) ... # Conflicts: # docs/content/docs/v4/cookbook/advanced/child-workflows.mdx # docs/content/docs/v5/cookbook/advanced/child-workflows.mdx # docs/content/docs/v5/cookbook/agent-patterns/durable-agent.mdx
pranaygp added a commit
that referenced
this pull request
Jun 11, 2026
* origin/main: [core] V2: unify wait+step queue dispatch in suspension processing (#1925) fix(world-local,world-postgres): make duplicate hook_created idempotent (#2295) docs(observability): remove MVP implementation detail bullet (#2367) fix: settle aborted parallel steps before completing abortParallelWorkflow (#2244) Add native v4 workflow attribute events (#2226) Version Packages (beta) (#2326) [ci] Fix flaky windows unit tests (#2359) Capture Vercel runtime logs when e2e Vercel Prod lanes fail (#2356) Fix e2e failure reporting under vitest 4 and preserve fetch error causes (#2355) [core] Fix process crash from rejected waitUntil promises (#2336) [core] Remove duplicate `waitUntil` for suspension handler async operations (#2345) Prevent local tests from hanging (#2338) feat(core): add optional namespace for queue topic prefix (#2305) [web-shared] Show precise durations in the new trace viewer (#2335) Validate unique workflow step IDs at build time (#2018) Move run attributes into their own detail card (#2327) [core] Forward-port stream reconnect to getReadable level (#2318) [docs] Add "Step executed multiple times" error page (#2310) Fix flickering on the detail panel when navigating the trace viewer (#2325)
VaguelySerious added a commit
that referenced
this pull request
Jun 22, 2026
Addresses review feedback on #2318: a transient failure of the reopen itself (`connect()` throwing inside `reconnect()`) was fatal — only `reader.read()` errors were retried. A brief server unavailability during the reconnect window is exactly the blip this wrapper exists to survive, so count it against the reconnect budget and retry rather than erroring the stream. Budget exhaustion (a server that stays down) still terminates. Adds a test: a reopen that throws once then succeeds recovers transparently. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Forward-port of #1847 (which targets
stable) ontomain.Same mechanism: object/serialized streams now transparently auto-reconnect at the getReadable level. When the upstream server stream connection errors mid-run (e.g. a per-session duration timeout), the wrapper reopens the stream at
startIndex + consumedFrames, counting 4-byte length-prefixed wire frames to compute the resume position. Partial in-flight frame bytes are discarded and resent in full from the new index. A clean EOF (no error) is treated as stream completion and does not reconnect. NegativestartIndex(last-N) skips reconnect and behaves as a single-shot read.Adapted to
main's world API: reads go throughworld.streams.get(runId, name, startIndex), sorunIdis threaded intocreateReconnectingFramedStream.Bounded by a consecutive-failure cap (resets on forward progress) and an absolute total-reconnect backstop so a misbehaving backend can never loop forever.
Object streams only. Byte-stream reconnect is a separate follow-up since it depends on byte-stream framing (#1853).
🤖 Generated with Claude Code