Verify stream completion on clean EOF in the reconnecting framed reader - #3564

Merged
alangenfeld merged 2 commits into
mainfrom
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged

Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom
alangenfeld/stream-eof-verify

Conversation

@alangenfeld

@alangenfeldalangenfeld commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary & Motivation

Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.

Test Plan

Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.

@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot

changeset-botBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5daa34b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm

@github-actions

github-actionsBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

✅ All tests passed

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R

E2E Test Summary

Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category

✅ ▲ Vercel Production

AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028

✅ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90128

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28

Streams

Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)

 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)

Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)

2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):

control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):

sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):

control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms
📜 Previous results (1)

6e2b0a5

Fri, 14 Aug 2026 22:18:17 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28

Streams

Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology

Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.

The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 Mint-ordered log — 3 fail of 41 total

log=mint-ordered · fence=per-spec

scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0

Full trace: world-sim-mint.txt

🟢 Append-only log — 0 fail of 41 total

log=append-only · fence=per-spec

scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0

Full trace: world-sim-append-only.txt

VaguelySerious
VaguelySerious approved these changes Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md Outdated
@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld marked this pull request as ready for review August 14, 2026 22:54
@alangenfeld
alangenfeld requested a review from a team as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld merged commit c5f9177 into mainAug 17, 2026
429 of 434 checks passed
@alangenfeld
alangenfeld deleted the alangenfeld/stream-eof-verify branch August 17, 2026 17:58
@github-actionsgithub-actionsBot mentioned this pull request Aug 17, 2026
Closed
github-actionsBot added a commit that referenced this pull request Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot mentioned this pull request Aug 17, 2026
Merged
@github-actions

github-actionsBot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Backport to stable failed for c5f9177 due to a workflow error (backport job run).

This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.

Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

c5f9177be3c58a2b5616f798932061b67e0b54a5

This was referenced Aug 17, 2026
Closed
Open
Closed
Open
Open
Merged
pranaygp added a commit that referenced this pull request Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot mentioned this pull request Aug 24, 2026
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Verify stream completion on clean EOF in the reconnecting framed reader - #3564
Merged
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Aug 17, 2026
Merged
Verify stream completion on clean EOF in the reconnecting framed reader#3564
alangenfeld merged 2 commits into
mainfrom 
alangenfeld/stream-eof-verify
Conversation
@alangenfeld
@alangenfeldalangenfeld
commented
Aug 14, 2026

Copy link


Copy Markdown

Collaborator
Summary & Motivation
Long live reads were ending silently at the server's 2-minute connection cap: the max-duration abort reaches the client as a clean EOF on some transport paths, and the reader read that as end-of-stream. On EOF it now consults streams.getInfo and reconnects from the next chunk unless the stream is done and every chunk up to the tail was delivered. A failed metadata read trusts the EOF, so a transient blip can't fail a healthy completion.
Test Plan
Tests added for the reconnect, verified-completion, metadata-failure, and reconnect-budget paths.
@alangenfeld
Verify stream completion on clean EOF in the reconnecting framed reader
6e2b0a5
A clean EOF was trusted as genuine end-of-stream. In production the
server's v3 max-duration abort is normalized into a graceful EOF on the
way to the client, so long-lived reads ended silently at the 2-minute
connection cap: zero reconnects across all reads org-wide, with read
completions clustering at exactly 120s. A completed stream cut mid-body
had the same shape - a clean EOF short of the tail.
On EOF the reader now consults streams.getInfo and treats the EOF as a
cut unless the stream is done AND the frames delivered cover every chunk
up to the tail, reconnecting from the next chunk exactly like an errored
connection. A metadata failure falls back to trusting the EOF so a
transient blip cannot fail a healthy completion. Negative-startIndex
(tail) reads keep their single-shot behavior.
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@changeset-bot
changeset-botBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daa34b
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 10:50pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 10:50pm
example-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-python-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 10:50pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 10:50pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 10:50pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 10:50pm
workflow-webReadyReadyPreviewAug 14, 2026 10:50pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
  • run-pickup-stall · addTenWorkflow (tanstack-start) · at 22:49:21Z · abandoned wrun_01M017DYGX7F9JM10KD32X9P8R
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production347407384212
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ 🌐 Cross-language Conformance90128137
✅ vercel-multi-region270027
Total152520254017792
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ python-node80148
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ 🌐 Cross-language Conformance
AppPassedFailedSkipped
✅ python90128
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daa34b · Fri, 14 Aug 2026 23:11:00 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1292 (+452%) 🔻1373 🔴 (+21%) 🔻1390 🔴 (+17%) 🔻1417 🔴 (-20%) 💚30
TTFSstream236 (+3.1%)1351 🔴 (+24%) 🔻1374 🔴 (+20%) 🔻1485 🔴 (-2.4%)30
TTFShook + stream1559 (+282%) 🔻1645 🔴 (+20%) 🔻1694 🔴 (+21%) 🔻1889 🔴 (+22%) 🔻30
Fan-out TTFSPromise.all(100 steps)8150 (-13%)9610 (-8.0%)10841 (-4.8%)15746 (-57%) 💚10
Fan-out TTLSPromise.all(100 steps)17876 (-6.3%)22515 (±0%)22987 (-0.7%)28817 (-38%) 💚10
STSO1020 steps (inline)117 (-18%) 💚429 (-13%)461 (-15%) 💚612 (-14%)1018
STSO1020 steps (queue-hop)2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚2072 (-29%) 💚1
WO1020 steps420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)420193 (-5.2%)1
CRTTfirst chunk (pooled)82 (-22%) 💚114 (-42%) 💚176 (-79%) 💚251 (-72%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)101 (-1%)5 (±0%)5 (-2%)106 (-34%)143 (-43%)214 (-39%)418 (-55%)129 (-31%)10
size sweep (100/s, 160B-12KB)100 (±0%)101 (+3%)334 (±0%)335 (+3%)90 (-27%)120 (-69%)251 (-60%)624 (-35%)165 (-55%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)89.3 (±0%)16.2 (±0%)16.2 (±0%)118 (-6%)109 (-34%)161 (-38%)715 (-19%)214 (-70%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)183 (+32%)120 (-31%)158 (-30%)482 (+33%)343 (+1%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (±0%)710 (±0%)710 (±0%)94 (-54%)152 (-55%)246 (-54%)491 (-39%)293 (-22%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 440013ms → this run 417847ms (Δ -22166ms, -5%)
 100-150 ms ┃ main 1 this 13 +12
150-200 ms █┃█ main 33 this 29 -4
200-250 ms █░┃ main 15 this 40 +25
250-300 ms ████░░░░░┃ main 49 this 116 +67
300-350 ms ███████████░░░░┃ main 129 this 191 +62
350-400 ms ████████░░░░░░░░░┃ main 90 this 207 +117
400-450 ms ███████████████████░░░░┃ main 227 this 282 +55
450-500 ms ████████┃█████████████ main 260 this 108 -152
500-550 ms ┃██████████ main 128 this 15 -113
550-600 ms ┃███ main 43 this 5 -38
600-650 ms ┃█ main 18 this 3 -15
650-700 ms ┃ main 11 this 0 -11
700-750 ms ┃ main 6 this 0 -6
750-800 ms ┃ main 2 this 0 -2
800-850 ms ┃ main 3 this 0 -3
850-900 ms ┃ main 2 this 0 -2
950-1000 ms ┃ main 1 this 0 -1
2500-2550 ms ┃ main 0 this 1 +1
2650-2700 ms ┃ main 0 this 2 +2
5100-5150 ms ┃ main 0 this 1 +1
5350-5400 ms ┃ main 0 this 2 +2
5650-5700 ms ┃ main 0 this 1 +1
7400-7450 ms ┃ main 0 this 1 +1
9900-9950 ms ┃ main 0 this 1 +1

1020 steps (queue-hop)
Cumulative STSO time: main 2925ms → this run 2072ms (Δ -853ms, -29%)
2000-2500 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
2500-3000 ms ┃███████████████████████ main 1 this 0 -1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ······█▇▁···· 109.3 (-40%) 99 (-27%) 214 (-39%) 418 (-55%) 3000
sweep ·····▁█▅▁▁··· 108.9 (-50%) 93 (-37%) 251 (-60%) 624 (-35%) 3000
gw 1x ·····▁█▄▁▁··· 103.3 (-30%) 92 (-27%) 161 (-38%) 715 (-19%) 5295
eve 1x ·····▁█▆▁▁··· 105.7 (-26%) 91 (-27%) 158 (-30%) 482 (+33%) 5186
eve 2x ·····▁▇█▂▁··· 129 (-42%) 107 (-40%) 246 (-54%) 491 (-39%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control █▄▄█▃▂▃▃▂▁ 94–134ms
sweep █▁▁▃▃▇█▇▂▁ 96–125ms
gw 1x ▅▃▁▂▂▁▁▂▂█ 88–152ms
eve 1x █▂▂▁▁▂▃▃▃▁ 87–175ms
eve 2x ▂▁▁▁▁▁▄█▄▃ 102–217ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep █▇▅▂▆▄▁ 107–110ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▅▆█▆▂▃▆▂▁ 27–37ms
sweep ▂▄▅▅▇▃█▃▂▁ 33–46ms
gw 1x █▄▂▁▄▂▂▄▁█ 24–34ms
eve 1x █▃▁▂▂▂▄▃▂▃ 19–33ms
eve 2x ▄▃▁▂▄▂▅▄▆█ 17–27ms

📜 Previous results (1)
6e2b0a5
Fri, 14 Aug 2026 22:18:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep219 (-6.4%)1417 🔴 (+25%) 🔻1488 🔴 (+25%) 🔻1541 🔴 (-13%)30
TTFSstream256 (+12%)1422 🔴 (+31%) 🔻1455 🔴 (+27%) 🔻1516 🔴 (±0%)30
TTFShook + stream405 (-0.7%)1725 🔴 (+26%) 🔻1743 🔴 (+25%) 🔻1778 🔴 (+15%) 🔻30
Fan-out TTFSPromise.all(100 steps)9384 (±0%)10778 (+3.2%)10904 (-4.2%)10964 (-70%) 💚10
Fan-out TTLSPromise.all(100 steps)18031 (-5.5%)19688 (-13%)19896 (-14%)20264 (-56%) 💚10
STSO1020 steps (inline)140 (-2.1%)460 (-6.3%)509 (-6.4%)697 (-1.8%)1018
STSO1020 steps (queue-hop)2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚2283 (-22%) 💚1
WO1020 steps409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)409244 (-7.6%)1
CRTTfirst chunk (pooled)106 (+1.0%)173 (-12%)240 (-72%) 💚322 (-64%) 💚28
Streams
Scenariowr c/srd c/swr KiB/srd KiB/sCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)100 (±0%)99.3 (-2%)5 (±0%)5 (-3%)122 (-24%)305 (+21%)846 (+142%)1186 (+26%)562 (+199%)10
size sweep (100/s, 160B-12KB)100 (±0%)90.7 (-7%)333 (±0%)302 (-7%)159 (+29%)326 (-16%)574 (-8%)972 (+2%)626 (+72%)10
replay gateway-gpt-5.4-nano-2000t (1x)89.2 (±0%)88.9 (±0%)16.2 (±0%)16.2 (±0%)117 (-6%)617 (+274%)2921 (+1023%)4475 (+406%)755 (+6%)3
replay eve-gpt-5.6-sol-2000t (1x)54.7 (±0%)54.7 (±0%)355 (±0%)355 (±0%)232 (+68%)188 (+7%)344 (+53%)789 (+117%)486 (+43%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (±0%)109 (-1%)710 (±0%)708 (-1%)188 (-9%)346 (+3%)538 (±0%)944 (+17%)693 (+83%)3
ℹ️ Metric definitions & methodology
Streams: writer/reader sustained rates (steady window, 10% trimmed each side), first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
VaguelySerious
VaguelySerious
approved these changes
Aug 14, 2026
View reviewed changes
Comment thread.changeset/stream-eof-verify.md
Outdated

@alangenfeld
Tighten stream EOF changeset wording
5daa34b
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
@alangenfeld
alangenfeld
marked this pull request as ready for review
August 14, 2026 22:54
@alangenfeld
alangenfeld
requested a review
from a team
as a code ownerAugust 14, 2026 22:54
@alangenfeld
alangenfeld
merged commit c5f9177
into
mainAug 17, 2026

429 of 434 checks passed

@alangenfeld
alangenfeld
deleted the

alangenfeld/stream-eof-verify

branch
August 17, 2026 17:58
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Closed

github-actionsBot
added a commit
that referenced
this pull request
Aug 17, 2026
@github-actions
Verify stream completion on clean EOF in the reconnecting framed read…
ff5613e
…er (#3564)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 17, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 17, 2026

Copy link


Copy Markdown

Contributor
Backport to stable failed for c5f9177 due to a workflow error (backport job run).
This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.
Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
c5f9177be3c58a2b5616f798932061b67e0b54a5


This was referenced Aug 17, 2026
 Closed

 Open

 Closed

 Open

 Open

 Merged

pranaygp added a commit
that referenced
this pull request
Aug 24, 2026
@github-actions@pranaygp
Verify stream completion on clean EOF in the reconnecting framed read…
853a316
…er (#3564) (#3588)
Long live reads were ending silently at the server's 2-minute connection
cap: the max-duration abort reaches the client as a clean EOF on some
transport paths, and the reader read that as end-of-stream. On EOF it
now consults `streams.getInfo` and reconnects from the next chunk unless
the stream is done and every chunk up to the tail was delivered. A
failed metadata read trusts the EOF, so a transient blip can't fail a
healthy completion.
Tests added for the reconnect, verified-completion, metadata-failure,
and reconnect-budget paths.
---------
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>
Signed-off-by: Alex Langenfeld <alexlangenfeld@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pranay Prakash <pranay.gp@gmail.com>
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 24, 2026
 Merged

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@alangenfeld@VaguelySerious