Uh oh!
There was an error while loading. Please reload this page.
[world-vercel] Make HTTP/2 actually multiplex on the events path - #3190
Conversation
The events agent sets `allowH2: true` and does negotiate ALPN h2, but it never multiplexed: 16 concurrent event writes produced 8 in-flight requests over 8 TCP connections — byte-for-byte identical to the HTTP/1.1 default agent. Three independent undici gates each capped in-flight requests at one per connection, and all three had to be lifted: 1. `pipelining: 1`, inherited from the shared base options. undici gates in-flight requests on `client[kPipelining] ?? defaultPipelining ?? 1`; `client-h2.js` sets `defaultPipelining: Infinity`, but the Client constructor always coerces `pipelining` to a number, so that value is unreachable (nodejs/undici#4143). 2. `client-h2.js` reports the connection busy for any request with `idempotent === false`, and only GET/HEAD are idempotent by default — every event write is a POST (nodejs/undici#5390). 3. `client-h2.js` also reports busy for a stream/async-iterable body. events-v4 passes a materialized `Uint8Array`, but it dispatches via global `fetch` (deliberately, for outgoing-request observability), which streamifies every body. `pipelining` is split out of the shared base so each agent sets it explicitly, and an interceptor marks events requests idempotent and re-buffers small streamed bodies. Marking them idempotent affects concurrency only: RetryAgent's default `methods` excludes POST, so an event write is still never resent. Stream write/close move to their own options and deliberately do NOT multiplex: appends are not idempotent, and STREAM_RETRY_OPTIONS retries PUT on transient connection errors, so multiplexing would let one reset fail — and then re-send — several already-applied appends at once. Verified against api.vercel.com, 16 concurrent POSTs after warming the pool: events 0 new connections (was 7), stream writes 7, WORKFLOW_H2_MULTIPLEX=0 back to 7.
🦋 Changeset detectedLatest commit: 6883125 The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (1)b9f271dWed, 29 Jul 2026 19:28:19 GMT · run logs
ℹ️ Metric definitions & methodologyBest/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
🧪 E2E Test Results❌ Some tests failed ❌ Failed E2E Tests📦 Local Production (2 failed)nextjs-turbopack-stable (1 failed):
nextjs-webpack-stable (1 failed):
E2E Test SummarySummary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
❌ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
Uh oh!
There was an error while loading. Please reload this page.
karthikscale3
left a comment
There was a problem hiding this comment.
Reviewed the HTTP/2 multiplexing implementation, retry/body handling, stream-write isolation, fallback switch, and regression coverage. No blocking correctness issues found.
Co-authored-by: Nathan Rajlich <n@n8.io> Signed-off-by: Peter Wielander <mittgfu@gmail.com>
Uh oh!
There was an error while loading. Please reload this page.
No backport to This is a performance/throughput optimization (making the events agent actually multiplex over HTTP/2) plus new API surface ( To override, re-run the Backport to stable workflow manually via |
Backport to To resolve manually, push a backport branch and open a PR against git fetch origin stable
git checkout -b backport/pr-3190-to-stable origin/stable
git cherry-pick -S 34975f6b7dd8e0dad874e852eac04c9652f5971d # -S signs the commit# Fix conflicts, then:
git add -A
git cherry-pick --continue
git push -u origin backport/pr-3190-to-stable
gh pr create --base stable --head backport/pr-3190-to-stable \
--title "Backport #3190: <original PR title>" \
--body "Manual backport of #3190 (cherry-pick 34975f6b7dd8) to \`stable\`." |
`vercel.json`'s `env` block is documented as passing variables to the functions, but Vercel marks the property deprecated, and a flag that silently fails to arrive would make a null result read as "multiplexing is not the cause" — the one outcome where being wrong is expensive. instrumentation.ts now supplies the value if the deployment did not, and records which source won so the probe route can report it. That also tells us where mitigation advice should point if #3190 owns the regression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
Problem
The events agent sets
allowH2: trueand does negotiate ALPNh2— but it never multiplexed. Measured againstapi.vercel.comwith a warmed pool, 16 concurrent event writes produced 8 in-flight requests over 8 TCP connections: byte-for-byte identical to the HTTP/1.1 default agent. The existing tests assertedEVENTS_AGENT_OPTIONS.allowH2 === trueand that ALPN negotiatesh2— both true, neither measures concurrency.Cause
Three independent undici gates each cap in-flight requests at one per connection. All three had to be lifted; any one of them left alone keeps the agent serialized.
pipelining: 1, inherited from the shared base options. undici gates in-flight requests per connection onclient[kPipelining] ?? httpContext.defaultPipelining ?? 1.client-h2.jssetsdefaultPipelining: Infinity, but theClientconstructor always coercespipeliningto a number, sokPipeliningis never nullish and H2'sInfinityis unreachable — nodejs/undici#4143.client-h2.jsreports the connection busy wheneverrequest.idempotent === false, and undici only treats GET/HEAD as idempotent by default. Every event write is a POST — nodejs/undici#5390.client-h2.jsalso reports busy for a stream / async-iterable body.events-v4hands over a materializedUint8Array, but it dispatches through the globalfetch— deliberately, since that is what keeps events traffic visible in the outgoing-requests view — andfetchstreamifies every body on the way down.Verified matrix (16 concurrent POSTs):
pipeliningalone → 8 in-flight.idempotent: truealone → 8 in-flight. Both, plus a buffered body → 16 in-flight, 0 new connections.Change
pipeliningis split out of the shared base options so each agent sets it explicitly — undici overloads that one option to mean both "H1 pipelining depth" and "max in-flight H2 streams per connection", and the two paths want opposite values. The H1 default keepspipelining: 1(it deadlocks the webhookrespondWithmechanism otherwise).busy()gates, and retries are governed solely byRetryAgent, whose defaultmethodsexcludes POST — so an event write is still never resent.RetryAgentrather than theAgentinside it.RetryHandlercaptures its own copy of the body up front (wrapRequestBody), so composing it inside would leave a retry re-iterating the stream the interceptor had already consumed, sending an empty body. Composed outside, the retry replays the drainedBuffer. There is a test for exactly this.STREAM_AGENT_OPTIONSand deliberately do not multiplex. Appends are not idempotent, andSTREAM_RETRY_OPTIONSretries PUT on transient connection errors — multiplexing would let a single RST_STREAM/GOAWAY fail, and then re-send, several already-applied appends at once. This preserves today's one-request-per-connection failure isolation. (Belt-and-braces: gate 2 already serializes PUTs, but pinningpipelining: 1means the safety property doesn't silently depend on that upstream detail.)WORKFLOW_H2_MULTIPLEX=0restores one request per connection, documented in Runtime Tuning.Verification
Against real
api.vercel.com, 16 concurrent POSTs after warming the pool:WORKFLOW_H2_MULTIPLEX=0New tests run a real H2 loopback server through the production factory functions and hold requests open on a barrier to observe peak concurrency: one asserts the events dispatcher reaches 16 concurrent streams with no new connections, one asserts stream writes stay bounded by the pool, one asserts a re-buffered body survives a retry intact, one covers the kill switch. Reverting either the
pipeliningvalue or the interceptor composition fails these tests.pnpm typecheckand all 309 world-vercel unit tests pass.