[core] Send the disposed hook's token from the QuickJS engine - #3773

Merged
VaguelySerious merged 2 commits into
mainfrom
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged

[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom
peter/quickjs-hook-dispose-token

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented Aug 24, 2026

Copy link
Copy Markdown
Member

The two engines disagree about what a disposal says.

suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:

consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};

quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.

Why it matters

A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.

world-vercel is the case in hand: it uses the token to release the hook and its token claim together.

The change

One field, from the operation the VM already populates:

...(op.token===undefined ? {} : {eventData: {token: op.token}}),

Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.

Tests

packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.

Verified the first case fails without the change:

× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.

🤖 Generated with Claude Code

@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented Aug 24, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm

@changeset-bot

changeset-botBot commented Aug 24, 2026•
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3873b69

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented Aug 24, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)

 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1
📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):

control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):

sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):

control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms
📜 Previous results (1)

f3d26ba

Mon, 24 Aug 2026 23:29:51 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.

The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 24, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

❌ Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (8 failed)

python-node (8 failed):

  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability

🌐 Cross-language Conformance (9 failed)

python (9 failed):

  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028

✅ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

❌ 🌐 Cross-language Conformance

AppPassedFailedSkipped
❌ python09132

✅ vercel-http-transport

AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

✅ vercel-ws-transport

AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028

📋 View full workflow run

@VaguelySerious
VaguelySerious marked this pull request as ready for review August 24, 2026 23:10
@VaguelySerious
VaguelySerious requested a review from a team as a code ownerAugust 24, 2026 23:10
@github-actions

github-actionsBot commented Aug 24, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

TooTallNate
TooTallNate reviewed Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
const disposals = await disposeWith({
type: 'hook_dispose',
correlationId: 'hook_01JCHOOK',
token: 'order-42-approve',

@TooTallNateTooTallNateAug 24, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.

@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.

Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:

toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});

And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.

Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:

× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.

Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.

TooTallNate
TooTallNate requested changes Aug 24, 2026
View reviewed changes

@TooTallNateTooTallNate left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.

@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious

VaguelySerious commented Aug 24, 2026

Copy link
Copy Markdown
MemberAuthor

(AI) E2E Required Check triage: red on the python lanes, which are red on main.

The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.

Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.

The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:

 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.

Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.

Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:

  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.

TooTallNate
TooTallNate approved these changes Aug 25, 2026
View reviewed changes

@TooTallNateTooTallNate left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.

@VaguelySerious
VaguelySerious merged commit 27cab14 into mainAug 25, 2026
177 of 180 checks passed
@VaguelySerious
VaguelySerious deleted the peter/quickjs-hook-dispose-token branch August 25, 2026 14:49
@github-actionsgithub-actionsBot mentioned this pull request Aug 25, 2026
Merged
@github-actions

github-actionsBot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

No backport to stable for 27cab14 (AI decision).

The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot mentioned this pull request Aug 25, 2026
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
[core] Send the disposed hook's token from the QuickJS engine - #3773
Merged
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Aug 25, 2026
Merged
[core] Send the disposed hook's token from the QuickJS engine#3773
VaguelySerious merged 2 commits into
mainfrom 
peter/quickjs-hook-dispose-token
Conversation
@VaguelySerious
@VaguelySeriousVaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

Member
The two engines disagree about what a disposal says.
suspension-handler.ts (node:vm) has always sent the hook's token on hook_disposed:
consthookDisposedEvent: CreateEventRequest={eventType: 'hook_disposed'asconst,specVersion: SPEC_VERSION_CURRENT,correlationId: queueItem.correlationId,eventData: {token: queueItem.token,},};
quickjs-entrypoint.ts sent no eventData at all — even though the pending operation already carries the token, and already uses it to order same-token hook operations so a dispose releases a token before a later hook's creation is validated against it.
Why it matters
A world may key a hook's token claim separately from the hook itself, in which case retiring a hook means releasing both, and it needs to know which token this disposal is for. Told nothing, such a world has to look the token up before it can act — and the same workflow then does measurably different work depending on which VM ran it. hook_disposed's eventData.token is optional in the schema, so neither engine is wrong on the wire; they are just inconsistent, which makes this the kind of thing that surfaces as a backend-shaped bug report rather than an engine one.
world-vercel is the case in hand: it uses the token to release the hook and its token claim together.
The change
One field, from the operation the VM already populates:
...(op.token===undefined ? {} : {eventData: {token: op.token}}),
Omitted rather than sent as undefined when the operation carries none. PendingHookDispose.token is optional, and an explicit undefined encodes differently across the JSON and CBOR wire formats — leaving the field off keeps this engine indistinguishable from a client too old to send one, which is a case a world already has to handle.
Tests
packages/core/src/runtime/quickjs-hook-dispose-token.test.ts drives the entrypoint over a single hook_dispose operation with the VM mocked, so what is under test is the request the entrypoint builds rather than the VM that produced the operation. Two cases: the token is carried, and eventData is absent entirely when the operation has none.
Verified the first case fails without the change:
× carries the disposed hook's token
AssertionError: expected undefined to deeply equal { token: 'order-42-approve' }

packages/core/src/runtime is green at 728 tests, and hookDisposeTestWorkflow already covers disposal end to end on the QuickJS lanes.
🤖 Generated with Claude Code
@VaguelySerious@claude
fix(core): send the disposed hook's token from the QuickJS engine
f3d26ba
The two engines disagreed about what a disposal says. The node:vm engine has
always sent `eventData: { token }` on `hook_disposed`; the QuickJS engine sent
no `eventData` at all, even though the pending operation already carries the
token and already uses it to order same-token hook operations.
That matters to a world that keys a hook's token claim separately from the hook
itself, because releasing both means knowing which token this disposal is for.
Without it such a world has to look the token up before it can act, and the
same workflow does different work depending on which VM ran it — a difference
that surfaces as a backend-shaped bug rather than an engine one.
The field is omitted rather than sent as `undefined` when the operation carries
no token, since `PendingHookDispose.token` is optional and the two wire formats
disagree about how to encode an explicit `undefined`. A world reading it cannot
then tell this engine from a client too old to send one, which is a case it
already handles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vercel
vercelBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-nextjs-workflow-webpackReadyReadyPreview, v0Aug 24, 2026 11:38pm
example-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-astro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-express-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-fastify-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-hono-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nestjs-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nitro-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-nuxt-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-python-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-sveltekit-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-tanstack-start-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workbench-vite-workflowReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-docsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-swc-playgroundReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-tarballsReadyReadyPreview, v0Aug 24, 2026 11:38pm
workflow-webReadyReadyPreview, v0Aug 24, 2026 11:38pm
@changeset-bot
changeset-botBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 3873b69
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 3873b69 · Mon, 24 Aug 2026 23:51:48 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1261 (+27%) 🔻1492 🔴 (+35%) 🔻1507 🔴 (+31%) 🔻1540 🔴 (+23%) 🔻30
TTFSstream461 (-35%) 💚1502 🔴 (+37%) 🔻1529 🔴 (+38%) 🔻1647 🔴 (+28%) 🔻30
TTFShook + stream1507 (+9.7%)1749 🔴 (+19%) 🔻1866 🔴 (+24%) 🔻2278 🔴 (+43%) 🔻30
Fan-out TTFSPromise.all(100 steps)579 (-11%)1991 (+17%) 🔻2115 (+18%) 🔻2215 (+14%)10
Fan-out TTLSPromise.all(100 steps)1789 (-7.4%)3701 (+17%) 🔻3718 (+13%)8728 (+2.9%)10
STSO1020 steps (inline)78 (-26%) 💚137 (-7.4%)160 (-4.8%)234 (+7.3%)1019
WO1020 steps138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)138942 (-5.9%)1
CRTTfirst chunk (pooled)91 (+3.4%)147 (+19%) 🔻237 (+23%) 🔻592 (-4.5%)28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)118 (+5%)118 (-9%)193 (-9%)535 (-30%)102 (-29%)10
size sweep (100/s, 160B-12KB)107 (-5%)166 (+17%)264 (+41%)829 (-26%)192 (+40%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)113 (-83%)148 (-96%)329 (-93%)199 (-8%)3
replay eve-gpt-5.6-sol-2000t (1x)136 (+15%)114 (-15%)151 (-11%)279 (-19%)261 (±0%)2
replay eve-gpt-5.6-sol-2000t (2x)141 (+15%)152 (-17%)216 (-18%)332 (-40%)214 (-6%)3
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 147417ms → this run 138798ms (Δ -8619ms, -6%)
 50-100 ms ┃ main 0 this 2 +2
100-150 ms ██████████████████████░┃ main 785 this 871 +86
150-200 ms ███┃██ main 216 this 131 -85
200-250 ms ┃ main 14 this 9 -5
250-300 ms ┃ main 4 this 3 -1
300-350 ms ┃ main 0 this 2 +2
350-400 ms ┃ main 0 this 1 +1

📈 CRTT drill-down vs main (RTT distributions & profiles)
variant RTT 1ms→5s+ avg p50 p90 p99 n
control ·····▁█▇▁▁··· 106.3 (-11%) 97 (-8%) 193 (-9%) 535 (-30%) 3000
sweep ······▆█▂▁··· 142 (+15%) 109 (-1%) 264 (+41%) 829 (-26%) 3000
gw 1x ·····▁█▆▁···· 101 (-70%) 96 (-5%) 148 (-96%) 329 (-93%) 5295
eve 1x ·····▁█▅▁···· 99.9 (-10%) 91 (±0%) 151 (-11%) 279 (-19%) 5186
eve 2x ·····▁▆█▂···· 120.4 (-14%) 111 (-14%) 216 (-18%) 332 (-40%) 7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):
control ▅▆▅▂▁▂▃█▃▄ 95–123ms
sweep ▆▇█▄▆▂▁▂▁▂ 108–188ms
gw 1x ▆▃▂█▄▃▂▂▂▁ 90–125ms
eve 1x ▄▃▄▅▁▂▇█▃▃ 87–116ms
eve 2x ▅▁▂▇▂▂▇█▆▅ 103–137ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):
sweep ▇█▅▂▄▂▁ 140–145ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):
control ▁▆▄▃▃▁█▄▂▄ 27–42ms
sweep ▁█▅▃▆▄▄▄▄▂ 35–80ms
gw 1x █▄▂▇▄▄▄▆▁▄ 26–32ms
eve 1x ▂█▃▄▃▁▇▄▃█ 18–25ms
eve 2x ▇▅▁▇▅▄▅▂▆█ 17–24ms

📜 Previous results (1)
f3d26ba
Mon, 24 Aug 2026 23:29:51 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1272 (+28%) 🔻1425 🔴 (+29%) 🔻1445 🔴 (+25%) 🔻1592 🔴 (+27%) 🔻30
TTFSstream325 (-54%) 💚1421 🔴 (+30%) 🔻1428 🔴 (+29%) 🔻1511 🔴 (+17%) 🔻30
TTFShook + stream708 (-48%) 💚1658 🔴 (+13%)1699 🔴 (+13%)2065 🔴 (+29%) 🔻30
Fan-out TTFSPromise.all(100 steps)522 (-20%) 💚863 (-50%) 💚1953 (+9.2%)2117 (+8.8%)10
Fan-out TTLSPromise.all(100 steps)1986 (+2.8%)6162 (+94%) 🔻9220 (+180%) 🔻9272 (+9.4%)10
STSO1020 steps (inline)90 (-14%)123 (-17%) 💚138 (-18%) 💚210 (-3.7%)1019
WO1020 steps123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚123839 (-16%) 💚1
CRTTfirst chunk (pooled)85 (-3.4%)162 (+31%) 🔻684 (+254%) 🔻4127 (+566%) 🔻28
Streams
ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)117 (+4%)125 (-4%)182 (-14%)305 (-60%)92 (-36%)10
size sweep (100/s, 160B-12KB)117 (+4%)138 (-3%)472 (+152%)5220 (+364%)155 (+13%)10
replay gateway-gpt-5.4-nano-2000t (1x)141 (+15%)114 (-83%)146 (-96%)255 (-95%)186 (-14%)3
replay eve-gpt-5.6-sol-2000t (1x)400 (+239%)113 (-16%)163 (-4%)364 (+6%)252 (-4%)2
replay eve-gpt-5.6-sol-2000t (2x)109 (-11%)1375 (+651%)2905 (+1005%)3720 (+575%)247 (+8%)3
ℹ️ Metric definitions & methodology
Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.
The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.
The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles
Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600
All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).
Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (8 failed)
python-node (8 failed):
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF | 🔍 observability
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR | 🔍 observability
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS | 🔍 observability
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1 | 🔍 observability
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT | 🔍 observability
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53 | 🔍 observability
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96 | 🔍 observability
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0 | 🔍 observability
🌐 Cross-language Conformance (9 failed)
python (9 failed):
  • deploymentId: 'latest' is a no-op in non-Vercel worlds | wrun_01M0V2K3WJEPP37AE5W5J4E91J
  • promiseAllWorkflow | wrun_41M0V26RGQ0GYZWGMJ6ASX7VVF
  • sleepingWorkflow | wrun_41M0V27FNX0GSJ6WM8FB5C49FR
  • parallelSleepWorkflow | wrun_41M0V27H550GN307C8TV5KRCQS
  • nullByteWorkflow | wrun_41M0V27QQA0GYPFBE9X7S5XAE1
  • cancelRun - cancelling a running workflow | wrun_41M0V2CJRH0GTP139S1ZWH40FT
  • cancelRun via CLI - cancelling a running workflow | wrun_41M0V2CNKA0GPFGVK7Y6SC4V53
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_41M0V2CYA70GR2EVTGHRAKMT96
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_41M0V2DGW80GXKV8CDSJ5Q6KZ0
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • abortDeterministicBranchWorkflow: if-check takes same path on first-run and replay (nextjs-turbopack)
  • addTenWorkflow (nextjs-webpack)
  • addTenWorkflow (nitro)
  • cancelRun via CLI - cancelling a running workflow (fastify)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nitro)
  • deploymentId: 'latest' is a no-op in non-Vercel worlds (nuxt)
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (nest)
  • utf8StreamWorkflow (hono)
  • webhookWorkflow (nextjs-webpack)
🛠 Infra Events (absorbed by the harness)
Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.
33 infra events
  • cold-start-warmup · suite warmup (python) · at 23:40:10Z · abandoned wrun_41M0V26ZST0GY2P6EJ9TAXDF2T · (+7 more)
  • run-pickup-stall · nullByteWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMH00GN14PJQK04CZVDK
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGQ0GPMQ0N7575Z5GZJ
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAH
  • run-pickup-stall · sleepingWorkflow (python) · at 23:40:26Z · abandoned wrun_41M0V2AMGX0GY27F9CJ90E4WAG
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:27Z · abandoned wrun_41M0V2AMRM0GV0SMK4R40WVEH4
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:40:59Z · abandoned wrun_41M0V2BM7E0GWHQRBWPFDZ332S
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFP80GQKD1Q57AVR1D21
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFRZ0GGG642B0HRMGHRW
  • run-pickup-stall · nullByteWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFVJ0GKJPFBVX5T9WCPT
  • run-pickup-stall · sleepingWorkflow (python) · at 23:41:27Z · abandoned wrun_41M0V2CFZK0GRF3TTEMSAT9CT8
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:42:17Z · abandoned wrun_41M0V2CKN70GXKD3TDEX89QNJY
  • cold-start-warmup · suite warmup (tanstack-start) · at 23:42:32Z · abandoned wrun_01M0V2E4HXFQ86A9QRCY6KTVP1
  • cold-start-warmup · suite warmup (python) · at 23:43:33Z · abandoned wrun_01M0V2D5F3QFCK8MK0106TGE6B · (+7 more)
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9A
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKYNA82S50HNZWZZE7Q
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKPWDV69YGNHC684T9B
  • run-pickup-stall · sleepingWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTKWM1WHPR2G4VJB5HS7
  • run-pickup-stall · nullByteWorkflow (python) · at 23:43:48Z · abandoned wrun_01M0V2GTM0Z9VWC5Y6JSWMNZ00
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:44:38Z · abandoned wrun_41M0V2F7JT0GKYY29C3Y457115
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:44:38Z · abandoned wrun_41M0V2FBHR0GS4CW6PXGYBW4PW
  • run-pickup-stall · deploymentId: 'latest' is a no-op in non-Vercel worlds (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7GYW7HRGG7AT00P889
  • run-pickup-stall · parallelSleepWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7NYSH02V126SH3GCS7
  • run-pickup-stall · sleepingWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7J79BBBW2WJ3HA8XG9
  • run-pickup-stall · promiseAllWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7KCB2HJ153EJ6DCD81
  • run-pickup-stall · nullByteWorkflow (python) · at 23:44:48Z · abandoned wrun_01M0V2JN7RP0PR3D22DWDWB319
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVABD7EEVPSE1FS9SM1
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:45:48Z · abandoned wrun_01M0V2MFVEYYGKH65Q5EBJTXD1
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:45:48Z · abandoned wrun_01M0V2MFV64C0MMQQ258PWDS08
  • run-pickup-stall · cancelRun via CLI - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND7601ZCCTNRANNFVNN7
  • run-pickup-stall · cancelRun - cancelling a running workflow (python) · at 23:46:18Z · abandoned wrun_01M0V2ND75V5DNFRQ32DP81JNT
  • run-pickup-stall · sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration (python) · at 23:46:48Z · abandoned wrun_01M0V2PAEHX874WWV099XZ5P3H
  • run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 23:46:53Z · abandoned wrun_01M0V2PF95KM44HXJZ9EBWKS6R
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production357087424320
✅ 💻 Local Development392205584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
❌ 🌐 Cross-language Conformance09132141
✅ vercel-http-transport8170143960
✅ vercel-multi-region270027
✅ vercel-ws-transport553087640
Total1705317277819848
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node132028
✅ astro-quickjs132028
✅ example-node132028
✅ example-quickjs132028
✅ express-node132028
✅ express-quickjs132028
✅ fastify-node132028
✅ fastify-quickjs132028
✅ hono-node132028
✅ hono-quickjs132028
✅ nest-node132028
✅ nest-quickjs132028
✅ nextjs-turbopack-node15703
✅ nextjs-turbopack-quickjs15703
✅ nextjs-webpack-node15703
✅ nextjs-webpack-quickjs15703
✅ nitro-node132028
✅ nitro-quickjs132028
✅ nuxt-node132028
✅ nuxt-quickjs132028
❌ python-node08152
✅ sveltekit-node15109
✅ sveltekit-quickjs15109
✅ tanstack-start-node132028
✅ tanstack-start-quickjs132028
✅ vite-node132028
✅ vite-quickjs132028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000
❌ 🌐 Cross-language Conformance
AppPassedFailedSkipped
❌ python09132
✅ vercel-http-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ hono132028
✅ nextjs-turbopack15703
✅ nitro132028
✅ vite132028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
✅ vercel-ws-transport
AppPassedFailedSkipped
✅ example132028
✅ express132028
✅ nextjs-turbopack15703
✅ vite132028
📋 View full workflow run
@VaguelySerious
VaguelySerious
marked this pull request as ready for review
August 24, 2026 23:10
@VaguelySerious
VaguelySerious
requested a review
from a team
as a code ownerAugust 24, 2026 23:10
@github-actions
github-actionsBot
commented
Aug 24, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 world-sim scenario book — 1 fail of 41 total
fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
❌ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim.txt
TooTallNate
TooTallNate
reviewed
Aug 24, 2026
View reviewed changes
Comment threadpackages/core/src/runtime/quickjs-hook-dispose-token.test.ts
 const disposals = await disposeWith({
 type: 'hook_dispose',
 correlationId: 'hook_01JCHOOK',
 token: 'order-42-approve',
@TooTallNateTooTallNateAug 24, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: This test injects a token directly into the pending operation, but the terminal system-hook cleanup path in quickjs-runtime.ts (the synthesized hook_dispose around lines 2621-2625) does not copy p.token. As a result, completing a workflow with a durable, unaborted AbortController hook still reaches this entrypoint without a token and emits no eventData, unlike node:vm. Please include token: p.token in that synthesized operation and cover that producer path. A shared cross-runtime conformance harness can remain a follow-up; fixing this focused regression does not require that refactor.
@VaguelySeriousVaguelySeriousAug 24, 2026

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in 3873b69b23. You are right about the mechanism and about the parity: I fixed the consumer and missed a producer.
Two paths build a hook_dispose. disposeHook in the VM builds it from the hook’s own closure and has always carried the token. The completion drain synthesizes a fresh object for a durable system hook the workflow never aborted, and dropped it:
toDispose.push({type: "hook_dispose",correlationId: p.correlationId,token: p.token,// <- addedhasCreatedEvent: false,});
And node:vm does send a token in that same case, which I checked rather than assumed: workflow.ts:91-99 marks the live system hook disposed at completion and hands the same queue item to handleSuspension, which reaches disposeHook(queueItem) and its eventData: { token: queueItem.token }. So the asymmetry was real and specific to this path.
Covered at the producer rather than by injecting an op, since injecting one is what let this through in the first place. The new test in quickjs-runtime.test.ts runs a workflow that creates an AbortController, replays it with the hook durably created and its step completed so the drain actually runs, and asserts the synthesized operation carries the hook’s token:
× carries the system hook token into the completion-drain disposal
AssertionError: expected undefined to be 'abrt_01JGFJJZ00NJ87CBWD49MG5PQC'

That is the failure on the previous commit; it passes on this one. packages/core/src/runtime is green at 729.
Agreed on the conformance harness being the right long-term answer and out of scope here. Worth noting what would have caught this cheaply in the meantime: the two engines share no test that asserts they emit the same event data for the same workflow, and both of my tests are per-engine. If you want, I can open a follow-up issue describing the harness shape rather than leaving it in a review thread.
TooTallNate
TooTallNate
requested changes
Aug 24, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Request changes. The entrypoint now forwards a token when the pending operation contains one, but QuickJS terminal cleanup still synthesizes tokenless system-hook disposal operations, so parity with node:vm remains incomplete. The blocking producer-path issue is inline. A shared runtime conformance harness would be preferable long-term but is not required for this fix.
@VaguelySerious@claude
fix(core): carry the token through the completion-drain hook disposal
3873b69
The entrypoint change covered the consumer and missed a producer. Two paths
reach it with a `hook_dispose` operation: `disposeHook` in the VM, which builds
the op from the hook's own closure and has always carried the token, and the
completion drain, which synthesizes a fresh object for a durable system hook
the workflow never aborted — and dropped it.
So finishing a workflow holding a live AbortController still emitted a
`hook_disposed` with no `eventData`, which is exactly the case node:vm sends a
token for: its drain marks the queue item disposed and hands the same item to
`handleSuspension`, token included.
Covered at the producer rather than by injecting an op: the new test runs a
workflow that creates an AbortController, replays it with the hook durably
created and its step done so the drain actually runs, and asserts the
synthesized operation carries the hook's token. It fails on the previous commit.
Reported in review on #3773.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@VaguelySerious
VaguelySerious
commented
Aug 24, 2026

Copy link


Copy Markdown

MemberAuthor
(AI) E2E Required Check triage: red on the python lanes, which are red on main.
The two failing jobs are E2E Vercel Prod Tests (python - node) (8 failures) and E2E Python Conformance (9), and the failing tests are promiseAllWorkflow, sleepingWorkflow, parallelSleepWorkflow, nullByteWorkflow, cancelRun, cancelRun via CLI, sleepInLoopWorkflow and resilient start — basic workflows, none involving hooks or AbortController.
Both lanes fail on main at the same step (Run E2E Tests) in each of its last three runs: 32553968829, 32546699992, 32545905106. My branch is not behind — its base is 3c0d60be90, which is main's current head — so this is not the stale-checkout failure these lanes usually have.
The diff is also the wrong shape to cause it. Five files, all additive, all QuickJS-engine or its tests:
 .changeset/quickjs-hook-dispose-token.md | 5 +
packages/core/src/runtime/quickjs-entrypoint.ts | 7 ++
.../src/runtime/quickjs-hook-dispose-token.test.ts | 110 ++++
packages/core/src/runtime/quickjs-runtime.test.ts | 80 ++++
packages/core/src/runtime/quickjs-runtime.ts | 6 ++

python - node runs the node:vm engine, which touches neither changed file.
Not re-running: a lane that fails identically on main will fail again. What is green and relevant: Unit Tests (ubuntu-latest), Unit Tests (windows-latest), and packages/core/src/runtime locally at 729 tests including the two new ones.
Two things I have NOT attributed and am not claiming are fine — flagging them so nobody assumes I checked:
  • world-sim reports 1 fail of 41, in-flight-before-decision-counted. It shows violations: 0 and replay: ok, and it was already failing on the first commit of this PR, before the producer fix. I have not compared it against main.
  • The benchmark comment shows TTFS regressions. That workflow is separately flaky and non-required, and I have not looked into whether these numbers mean anything.
TooTallNate
TooTallNate
approved these changes
Aug 25, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI: Approved. The terminal QuickJS system-hook producer now preserves p.token, and the new test exercises the actual completion-drain path that previously bypassed the entrypoint-only regression test. A shared node:vm/QuickJS conformance harness remains desirable as a non-blocking follow-up.
@VaguelySerious
VaguelySerious
merged commit 27cab14
into
mainAug 25, 2026

177 of 180 checks passed

@VaguelySerious
VaguelySerious
deleted the

peter/quickjs-hook-dispose-token

branch
August 25, 2026 14:49
@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Merged

@github-actions
github-actionsBot
commented
Aug 25, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 27cab14 (AI decision).
The change is confined to the QuickJS engine (packages/core/src/runtime/quickjs-entrypoint.ts, quickjs-runtime.ts and their tests), and verification shows stable has no QuickJS runtime files at all — git ls-tree origin/stable -- packages/core/src/runtime/ matches nothing containing quickjs, while only the node:vm suspension-handler.ts (which already sends the token) exists there. There is nothing on stable for this fix to apply to, so backporting it would import a main-only engine rather than fix a maintenance-line defect.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
27cab14adcc6f748500fca19cf78feeb60a125e7

@github-actionsgithub-actionsBot
mentioned this pull request
Aug 25, 2026
 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
@pranaygppranaygppranaygp approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@pranaygp