Prototype WebSocket stream transport - #3359

Closed
alangenfeld wants to merge 1 commit into
mainfrom
alangenfeld/plan-ws-streams-client
Closed

Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom
alangenfeld/plan-ws-streams-client

Conversation

@alangenfeld

@alangenfeldalangenfeld commented Aug 5, 2026•
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.

The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.

How did you test your changes?

Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.

ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s

A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.

  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.

Docs Preview

PR Checklist - Required to merge

  • 📦 pnpm changeset was run to create a changelog for this PR
    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  • 🔒 DCO sign-off passes (run git commit --signoff on your commits)
  • 📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete

@changeset-bot

changeset-botBot commented Aug 5, 2026•
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 989e774

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 5, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm

@github-actions

github-actionsBot commented Aug 5, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

❌ Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (8 failed)

python-node (8 failed):

  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability

🌐 Cross-language Conformance (9 failed)

python (9 failed):

  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028

✅ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

❌ 🌐 Cross-language Conformance

AppPassedFailedSkipped
❌ python09132

✅ vercel-http-transport

AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

✅ vercel-ws-transport

AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028

📋 View full workflow run

vercel[bot]
vercelBot reviewed Aug 5, 2026
View reviewed changes

@vercelvercelBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.

Fix on Vercel

@alangenfeld
alangenfeldforce-pushed the alangenfeld/plan-ws-streams-client branch 2 times, most recently from fe1efd3 to 6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed to Preview – workflow-docs August 5, 2026 18:19 Inactive
@alangenfeld
alangenfeldforce-pushed the alangenfeld/plan-ws-streams-client branch from 6b3032f to b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed to Preview – workflow-docs August 5, 2026 18:21 Inactive
@github-actions

github-actionsBot commented Aug 10, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)

100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1
📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):

control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):

sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):

control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.

The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@alangenfeld
alangenfeldforce-pushed the alangenfeld/plan-ws-streams-client branch from 8682925 to c422066CompareAugust 19, 2026 18:16
@github-actions

github-actionsBot commented Aug 19, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Prototype WebSocket stream transport - #3359
Closed
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Closed
Prototype WebSocket stream transport#3359
alangenfeld wants to merge 1 commit into
mainfrom 
alangenfeld/plan-ws-streams-client
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
This exploratory client extends the generic WebSocket transport from #3084 and pairs with workflow-server PR #710. The opt-in WORKFLOW_STREAMS_TRANSPORT=ws mode covers stream writes, batched writes, close, live reads, explicit resume/end reasons, and wire-level cancellation while keeping event writes on the existing one-shot request path.
The explicit stream_end.reason improves resume correctness without adding a second path to createReconnectingFramedStream: the WS adapter presents one stable framed stream and removes HTTP abort-vs-EOF interpretation, partial-frame discard/counting, and client resume arithmetic from core. The cost moves into world-vercel, which now owns subscription routing, reconnect budgets, cancellation acknowledgement, and socket lifecycle. Append retries remain limited to failures known not to have been sent plus 429; idempotent close/cancel may also retry ambiguous transport failures and 5xx.
How did you test your changes?
Cross-repo E2E used WS client deployment dpl_ESkNDLV3Ft2k1ZEQQJeWFk9C5RkB against server PR #710 deployment dpl_57VtZSaMF8M43dPYQH9td5ZNfCbG. Both full 30-sample benchmark runs completed: WS and HTTP.
ScenarioHTTPWSChange
First chunk, average165.9 ms109.4 ms-34.1%
First chunk, p75 / p90 / p99177 / 211 / 316 ms111 / 128 / 337 msp99 includes one WS outlier
300 text chunks, average excess overhead166.2 ms145.4 ms-12.5%
300 structured chunks, average excess overhead170.2 ms133.3 ms-21.7%
Effective text / structured throughput94.75 / 94.63 chunks/s95.38 / 95.75 chunks/sproducer paced at 100/s
A representative 300-chunk HTTP run emitted 50 physical stream HTTP requests (2 reads, 46 writes, 2 closes). The WS shape emitted no per-operation HTTP requests after at most two reader/writer-process socket upgrades; the stream operations were multiplexed as frames.
  • @workflow/world-vercel: 25 test files, 449 tests passed; the post-wire-fix focused suite passed 83 tests.
  • Package build, changed-file Biome, and git diff --check passed. CI Biome, Ubuntu unit tests, docs checks, and tarball checks passed.
  • The canonical fixture matches the server copy byte-for-byte (ee59799126b9720a226c62609b9fa79c13a00fdd63f3fcdca0b86480191f4a04).
  • Full workspace typecheck/test were attempted but stopped at the SWC plugin because Cargo is unavailable locally. Full lint reached one unrelated baseline error plus existing warnings.
Docs Preview
PR Checklist - Required to merge
  •  📦 pnpm changeset was run to create a changelog for this PR

    • Use the correct semver bump type: patch for bug fixes, minor for new features, major for breaking changes.
    • Use pnpm changeset --empty if you are changing documentation or workbench apps
  •  🔒 DCO sign-off passes (run git commit --signoff on your commits)
  •  📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete
@changeset-bot
changeset-botBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 989e774
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelMinor
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 25, 2026 2:44pm
example-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-astro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-express-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-hono-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-python-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workbench-vite-workflowReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-docsBuildingBuildingPreview, v0Aug 25, 2026 2:44pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-tarballsReadyReadyPreview, v0Aug 25, 2026 2:44pm
workflow-webReadyReadyPreview, v0Aug 25, 2026 2:44pm
@github-actions
github-actionsBot
commented
Aug 5, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D | 🔍 observability
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX | 🔍 observability
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0WP8HWN53Y7M0Y9GDBZ9X1C
  • promiseAllWorkflow | wrun_41M0WP12YD0GX9Q9C07A8KS88D
  • sleepingWorkflow | wrun_41M0WP1NKY0GQH1GPYPBQVAMWT
  • parallelSleepWorkflow | wrun_41M0WP1QAP0GSB4E31T1X7GYCX
  • nullByteWorkflow | wrun_41M0WP1YXS0GSQ5CWMQ906XC26
  • cancelRun - cancelling a running workflow | wrun_41M0WP6JMH0GH1RGASVTZFC13R
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0WP6KKA0GVZPBDT5M8SNGS9
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0WP6VC60GJ44585P6B0VEJN
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0WP7CYD0GV2RTENV5JQWNPT
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • addTenWorkflow (express)
  • cancelRun via CLI - cancelling a running workflow (hono)
  • health check (CLI) - workflow health command reports healthy endpoints (nextjs-webpack)
  • promiseAllWorkflow (express)
  • promiseAllWorkflow (hono)
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication (nextjs-webpack)
  • utf8StreamWorkflow (express)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 14:43:40Z · abandoned wrun_41M0WNXAT40GPYHP79QP2VKQZW · (+7 more)
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRN0GJ66HN0GZS7ZPCJ
  • run-pickup-stall · sleepingWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRW0GT31GAD0B5418J4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:43:57Z · abandoned wrun_41M0WP106E0GKZGGRKF4X08X0S
  • run-pickup-stall · nullByteWorkflow (python) · at 14:43:57Z · abandoned wrun_41M0WP0ZRZ0GH32AMT7K0P6C81
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:44:30Z · abandoned wrun_41M0WP206C0GZYK8VZ68XK51MH
  • run-pickup-stall · sleepingWorkflow (python) · at 14:44:57Z · abandoned wrun_41M0WP2VD70GT4KZF87F42ERDN
  • run-pickup-stall · nullByteWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VHT0GTNGN0P6DNSRBM3
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VJP0GQ566HM2CMWVQQ3
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:44:58Z · abandoned wrun_41M0WP2VCR0GMBYAWBQ396WQG8
  • cold-start-warmup · suite warmup (tanstack-start) · at 14:45:27Z · abandoned wrun_01M0WP3D7G53R6EPX245DAHG29
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:45:48Z · abandoned wrun_41M0WP30DJ0GKV3AVR0VV3ZTD0
  • cold-start-warmup · suite warmup (python) · at 14:46:33Z · abandoned wrun_01M0WP2KGKX73R3B3RWSQ10Y6H · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DWZ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KS490WEY57YA2F3S7D
  • run-pickup-stall · sleepingWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KPFJQBYXBR86SGRSC5
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KHTMHGB9SH5M7S7DX0
  • run-pickup-stall · nullByteWorkflow (python) · at 14:46:48Z · abandoned wrun_01M0WP68KTSX1KYXVTV474PAZ3
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:47:30Z · abandoned wrun_41M0WP5K9A0GSB8NK29W4096Z0
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:47:30Z · abandoned wrun_41M0WP5RA40GX47VDZWG3173T3
  • run-pickup-stall · sleepingWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837MH5Q5WKNKFJGPZF0A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837H13ZR67159T9M48CB
  • run-pickup-stall · promiseAllWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837D9ZC1AFK3FQ8497R5
  • run-pickup-stall · nullByteWorkflow (python) · at 14:47:48Z · abandoned wrun_01M0WP837J1JXD5KH8ZTK92185
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 14:47:48Z · abandoned wrun_01M0WP837GS06RB2MKN9G99MJR
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XV5SMPWTAKARK45RPVV
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVF6T5TC14MR3JHPK8A
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:48:48Z · abandoned wrun_01M0WP9XVKEZ2GDM5EZDSFD9YA
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV73F77519VX1EF4RCS4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 14:49:18Z · abandoned wrun_01M0WPAV75HQ9JF519MJPERP0Y
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 14:49:48Z · abandoned wrun_01M0WPBREPDMWX676R0HFHWW71
  • run-pickup-stall · fibonacciWorkflow - recursive workflow composition via start() (nextjs-webpack) · at 14:51:41Z · abandoned wrun_01M0WPF6VJ5BSV09FGB7Y7GF8E
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
vercel[bot]
vercelBot
reviewed
Aug 5, 2026
View reviewed changes
@vercelvercelBot
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:
Send-callback transport errors in WsTransport.request() and subscribe() default to delivery: 'unknown' even though the frame was never transmitted, blocking the safe stream-write retry path.
Fix on Vercel
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
2 times, most recently
from
fe1efd3 to
6b3032fCompareAugust 5, 2026 18:19
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:19
Inactive

@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
6b3032f to
b0874a0CompareAugust 5, 2026 18:21
@vercel
vercelBottemporarily deployed
to
Preview – workflow-docs
August 5, 2026 18:21
Inactive

@github-actions
github-actionsBot
commented
Aug 10, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 989e774 · Tue, 25 Aug 2026 15:05:04 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1308 (+236%) 🔻1417 🔴 (+29%) 🔻1464 🔴 (+29%) 🔻1873 🔴 (+34%) 🔻30
TTFSstream468 (+167%) 🔻1422 🔴 (+28%) 🔻1449 🔴 (+28%) 🔻1610 🔴 (+39%) 🔻30
TTFShook + stream1297 (+127%) 🔻1670 🔴 (+23%) 🔻1698 🔴 (+7.8%)1933 🔴 (-58%) 💚30
Fan-out TTFSPromise.all(100 steps)714 (+14%)2278 (+108%) 🔻2305 (+48%) 🔻2379 (-52%) 💚10
Fan-out TTLSPromise.all(100 steps)2156 (+19%) 🔻4354 (+8.2%)4631 (+14%)9126 (-8.3%)10
STSO1020 steps (inline)120 (+7.1%)172 (+20%) 🔻197 (+19%) 🔻275 (+15%)1019
WO1020 steps163367 (+12%)163367 (+12%)163367 (+12%)163367 (+12%)1
CRTTfirst chunk (pooled)103 (+24%) 🔻151 (+7.9%)250 (-3.8%)433 (-38%) 💚28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)141 (+26%)139 (-8%)230 (-37%)533 (-21%)138 (+14%)10
size sweep (100/s, 160B-12KB)129 (+15%)161 (+27%)219 (+35%)577 (-12%)139 (+29%)10
replay gateway-gpt-5.4-nano-2000t (1x)135 (-24%)136 (-79%)167 (-92%)531 (-85%)243 (-38%)3
replay eve-gpt-5.6-sol-2000t (1x)124 (-11%)133 (-6%)180 (-5%)481 (-6%)368 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)154 (+9%)220 (+17%)307 (+13%)552 (-33%)239 (+32%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 144386ms → this run 163212ms (Δ +18826ms, +13%)
100-150 ms ████████████┃███████████ main 853 this 458 -395
150-200 ms ████░░░░░░░░┃ main 138 this 467 +329
200-250 ms █┃ main 19 this 72 +53
250-300 ms ┃ main 4 this 18 +14
300-350 ms ┃ main 5 this 3 -2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······▅█▁▁··· 122.8 (-3%) 109 (+3%) 230 (-37%) 533 (-21%) 3000
sweep ······▄█▁▁··· 129.5 (+10%) 117 (+18%) 219 (+35%) 577 (-12%) 3000
gw 1x ·····▁▆█▁▁··· 114.8 (-61%) 107 (-13%) 167 (-92%) 531 (-85%) 5295
eve 1x ·····▁▇█▁▁··· 120.3 (-1%) 106 (+4%) 180 (-5%) 481 (-6%) 5186
eve 2x ·····▁▂█▃▁··· 159 (+8%) 136 (+1%) 307 (+13%) 552 (-33%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▃▄▂▁▂▂▂▁▅ 101–170ms
sweep █▃▂▃▁▂▁▂▁▂ 111–195ms
gw 1x █▃▃▃▂▂▃▁▃▂ 100–151ms
eve 1x ▃▄▁█▂█▆▄▄▁ 106–139ms
eve 2x ▂▁▁▃▁▃▅██▄ 122–228ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▆█▅▃▁▁▂ 127–133ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁█▆▅▂▄▁▄▂▆ 32–44ms
sweep ▁▃██▅▆▆▆▇▆ 33–55ms
gw 1x █▄▅▃▃▂▆▂▁▁ 30–38ms
eve 1x ▃▆▁▆▂█▃▄▁▇ 21–28ms
eve 2x ▄▃▁▃▄▅▆▃▆█ 21–29ms

ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@alangenfeld
alangenfeldforce-pushed
the
alangenfeld/plan-ws-streams-client
branch
from
8682925 to
c422066CompareAugust 19, 2026 18:16
@github-actions
github-actionsBot
commented
Aug 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
@alangenfeld
Prototype WebSocket stream transport
989e774
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@vercelvercel[bot]vercel[bot] left review comments

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@alangenfeld