[ci] Retry e2e tests once in CI, keeping retried tests visible by alangenfeld · Pull Request #3530 · vercel/workflow

alangenfeld · 2026-08-13T22:26:40Z

Summary & Motivation

Over the last 10 days ~93 Tests runs needed a human to click re-run until green (some took 6 attempts), because one test losing a timing race fails a whole 20+ minute e2e matrix job.

retry: 1 in CI only, so races still reproduce locally while debugging.
Retried-then-passed tests stay visible everywhere a failure would have been: ::warning annotations, an e2e-flaky-*.json sidecar per job, and a flaky section in the step summary and PR comment with per-app occurrence counts.
event-log-race-repro and the benchmarks pin retry: 0 — their failures are the measurement.

Test Plan

Reporter smoke-tested against a synthetic retried test and the aggregation script against synthetic artifact directories in both modes; confirmed the two retry: 0 suites still collect.

Over the last 10 days ~93 Tests runs were manually re-run until green, some taking 6 attempts: the e2e suites drive real deployments, and a single test losing a timing race fails a whole 20+ minute matrix job. A CI-only vitest retry (retry: 1) absorbs those single-test races. beforeEach/afterEach hooks run per attempt, so suites with file-restore hooks (dev.test.ts) retry cleanly. Retries must not hide real races, so a retried-then-passed test stays visible everywhere a failure would have been: the github-reporter emits ::warning annotations and an e2e-flaky-*.json sidecar, every e2e job uploads it, and aggregate-e2e-results.js renders a 'Flaky E2E Tests (passed on retry)' section in both the per-job step summary and the PR comment, with per-app occurrence counts. Harnesses whose failures are themselves the signal pin retry: 0: event-log-race-repro (a pass runs the full configured budget) and benchmarks (a regression should not be papered over by a luckier second sample). Local runs keep retry at 0 so races reproduce while debugging. Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>

Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>

changeset-bot · 2026-08-13T22:26:45Z

🦋 Changeset detected

Latest commit: b40e1aa

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages

Name	Type
@workflow/core	Patch
@workflow/builders	Patch
@workflow/cli	Patch
@workflow/next	Patch
@workflow/nitro	Patch
@workflow/vitest	Patch
@workflow/web-shared	Patch
@workflow/web	Patch
workflow	Patch
@workflow/world-testing	Patch
@workflow/astro	Patch
@workflow/nest	Patch
@workflow/rollup	Patch
@workflow/sveltekit	Patch
@workflow/vite	Patch
@workflow/nuxt	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

vercel · 2026-08-13T22:26:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
example-nextjs-workflow-turbopack	Ready	Preview	Aug 13, 2026 10:30pm
example-nextjs-workflow-webpack	Ready	Preview	Aug 13, 2026 10:30pm
example-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-astro-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-express-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-fastify-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-hono-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-nestjs-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-nitro-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-nuxt-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-python-workflow	Error		Aug 13, 2026 10:30pm
workbench-sveltekit-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-tanstack-start-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workbench-vite-workflow	Ready	Preview	Aug 13, 2026 10:30pm
workflow-docs	Ready	Preview, v0	Aug 13, 2026 10:30pm
workflow-swc-playground	Ready	Preview	Aug 13, 2026 10:30pm
workflow-tarballs	Ready	Preview	Aug 13, 2026 10:30pm
workflow-web	Ready	Preview	Aug 13, 2026 10:30pm

github-actions · 2026-08-13T22:27:00Z

🧪 E2E Test Results

✅ All tests passed

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (vite)
hookCleanupTestWorkflow - hook token reuse after workflow completion (astro)
hookCleanupTestWorkflow - hook token reuse after workflow completion (sveltekit)
step throw of a non-Error value preserves it as cause on the wrapping FatalError (hono)

E2E Test Summary

Summary

	Passed	Skipped	Total
✅ ▲ Vercel Production	3466	590	4056
✅ 💻 Local Development	3810	558	4368
✅ 📦 Local Production	3810	558	4368
✅ 🐘 Local Postgres	3810	558	4368
✅ 🪟 Windows	312	0	312
✅ vercel-multi-region	27	0	27
Total	15235	2264	17499

Details by Category

✅ ▲ Vercel Production

App	Passed	Skipped
✅ astro-node	128	28
✅ astro-quickjs	128	28
✅ example-node	128	28
✅ example-quickjs	128	28
✅ express-node	128	28
✅ express-quickjs	128	28
✅ fastify-node	128	28
✅ fastify-quickjs	128	28
✅ hono-node	128	28
✅ hono-quickjs	128	28
✅ nest-node	128	28
✅ nest-quickjs	128	28
✅ nextjs-turbopack-node	153	3
✅ nextjs-turbopack-quickjs	153	3
✅ nextjs-webpack-node	153	3
✅ nextjs-webpack-quickjs	153	3
✅ nitro-node	128	28
✅ nitro-quickjs	128	28
✅ nuxt-node	128	28
✅ nuxt-quickjs	128	28
✅ sveltekit-node	147	9
✅ sveltekit-quickjs	147	9
✅ tanstack-start-node	128	28
✅ tanstack-start-quickjs	128	28
✅ vite-node	128	28
✅ vite-quickjs	128	28

✅ 💻 Local Development

App	Passed	Skipped
✅ astro-stable-node	130	26
✅ astro-stable-quickjs	130	26
✅ express-stable-node	130	26
✅ express-stable-quickjs	130	26
✅ fastify-stable-node	130	26
✅ fastify-stable-quickjs	130	26
✅ hono-stable-node	130	26
✅ hono-stable-quickjs	130	26
✅ nest-stable-node	130	26
✅ nest-stable-quickjs	130	26
✅ nextjs-turbopack-canary-node	137	19
✅ nextjs-turbopack-canary-quickjs	137	19
✅ nextjs-turbopack-stable-node	156	0
✅ nextjs-turbopack-stable-quickjs	156	0
✅ nextjs-webpack-canary-node	137	19
✅ nextjs-webpack-canary-quickjs	137	19
✅ nextjs-webpack-stable-node	156	0
✅ nextjs-webpack-stable-quickjs	156	0
✅ nitro-stable-node	130	26
✅ nitro-stable-quickjs	130	26
✅ nuxt-stable-node	130	26
✅ nuxt-stable-quickjs	130	26
✅ sveltekit-stable-node	149	7
✅ sveltekit-stable-quickjs	149	7
✅ tanstack-start-node	130	26
✅ tanstack-start-quickjs	130	26
✅ vite-stable-node	130	26
✅ vite-stable-quickjs	130	26

✅ 📦 Local Production

App	Passed	Skipped
✅ astro-stable-node	130	26
✅ astro-stable-quickjs	130	26
✅ express-stable-node	130	26
✅ express-stable-quickjs	130	26
✅ fastify-stable-node	130	26
✅ fastify-stable-quickjs	130	26
✅ hono-stable-node	130	26
✅ hono-stable-quickjs	130	26
✅ nest-stable-node	130	26
✅ nest-stable-quickjs	130	26
✅ nextjs-turbopack-canary-node	137	19
✅ nextjs-turbopack-canary-quickjs	137	19
✅ nextjs-turbopack-stable-node	156	0
✅ nextjs-turbopack-stable-quickjs	156	0
✅ nextjs-webpack-canary-node	137	19
✅ nextjs-webpack-canary-quickjs	137	19
✅ nextjs-webpack-stable-node	156	0
✅ nextjs-webpack-stable-quickjs	156	0
✅ nitro-stable-node	130	26
✅ nitro-stable-quickjs	130	26
✅ nuxt-stable-node	130	26
✅ nuxt-stable-quickjs	130	26
✅ sveltekit-stable-node	149	7
✅ sveltekit-stable-quickjs	149	7
✅ tanstack-start-node	130	26
✅ tanstack-start-quickjs	130	26
✅ vite-stable-node	130	26
✅ vite-stable-quickjs	130	26

✅ 🐘 Local Postgres

App	Passed	Skipped
✅ astro-stable-node	130	26
✅ astro-stable-quickjs	130	26
✅ express-stable-node	130	26
✅ express-stable-quickjs	130	26
✅ fastify-stable-node	130	26
✅ fastify-stable-quickjs	130	26
✅ hono-stable-node	130	26
✅ hono-stable-quickjs	130	26
✅ nest-stable-node	130	26
✅ nest-stable-quickjs	130	26
✅ nextjs-turbopack-canary-node	137	19
✅ nextjs-turbopack-canary-quickjs	137	19
✅ nextjs-turbopack-stable-node	156	0
✅ nextjs-turbopack-stable-quickjs	156	0
✅ nextjs-webpack-canary-node	137	19
✅ nextjs-webpack-canary-quickjs	137	19
✅ nextjs-webpack-stable-node	156	0
✅ nextjs-webpack-stable-quickjs	156	0
✅ nitro-stable-node	130	26
✅ nitro-stable-quickjs	130	26
✅ nuxt-stable-node	130	26
✅ nuxt-stable-quickjs	130	26
✅ sveltekit-stable-node	149	7
✅ sveltekit-stable-quickjs	149	7
✅ tanstack-start-node	130	26
✅ tanstack-start-quickjs	130	26
✅ vite-stable-node	130	26
✅ vite-stable-quickjs	130	26

✅ 🪟 Windows

App	Passed	Failed	Skipped
✅ nextjs-turbopack-node	156	0	0
✅ nextjs-turbopack-quickjs	156	0	0

✅ vercel-multi-region

App	Passed	Failed	Skipped
✅ nextjs-turbopack	27	0	0

📋 View full workflow run

github-actions · 2026-08-13T22:27:04Z

📊 Workflow Benchmarks

commit b40e1aa · Thu, 13 Aug 2026 22:47:55 GMT · run logs

Backend: vercel · app: nextjs-turbopack

Metric	Scenario	Best (ms)	P75 (ms)	P90 (ms)	P99 (ms)	Samples
TTFS	step	1261 (+585%) 🔻	1377 🔴 (+24%) 🔻	1485 🔴 (+22%) 🔻	1712 🔴 (+7.6%)	30
TTFS	stream	1289 (+444%) 🔻	1339 🔴 (+21%) 🔻	1382 🔴 (+22%) 🔻	1409 🔴 (+17%) 🔻	30
TTFS	hook + stream	460 (+20%) 🔻	1649 🔴 (+20%) 🔻	1729 🔴 (+24%) 🔻	1801 🔴 (+17%) 🔻	30
Fan-out TTFS	Promise.all(100 steps)	8796 (-2.5%)	9486 (-8.3%)	10515 (-12%)	14740 (-95%) 💚	10
Fan-out TTLS	Promise.all(100 steps)	17503 (-1.4%)	18992 (-7.4%)	20565 (-70%) 💚	24176 (-92%) 💚	10
STSO	1020 steps (inline)	118 (-22%) 💚	164 (-27%) 💚	180 (-30%) 💚	288 (-45%) 💚	1019
WO	1020 steps	168616 (-26%) 💚	168616 (-26%) 💚	168616 (-26%) 💚	168616 (-26%) 💚	1
SL	stream latency	86 (-8.5%)	126 🔴 (-16%) 💚	146 🔴 (-9.3%)	344 🔴 (-2.8%)	30
SO	stream overhead (text)	105 (-33%) 💚	161 (-46%) 💚	167 (-62%) 💚	218 (-85%) 💚	30
SO	stream overhead (structured)	106 (-10%)	164 (-60%) 💚	174 (-69%) 💚	296 (-88%) 💚	30

📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 228513ms → this run 168423ms (Δ -60090ms, -26%)

 100-150 ms ░░░░░░░░░░░░░░░░░░░┃ main 0 this 436 +436
150-200 ms ███████████████████████┃ main 512 this 536 +24
200-250 ms ┃████████████████ main 385 this 29 -356
250-300 ms ┃██ main 71 this 8 -63
300-350 ms ┃ main 22 this 5 -17
350-400 ms ┃ main 11 this 4 -7
400-450 ms ┃ main 5 this 0 -5
450-500 ms ┃ main 1 this 0 -1
500-550 ms ┃ main 4 this 0 -4
600-650 ms ┃ main 1 this 0 -1
650-700 ms ┃ main 1 this 0 -1
900-950 ms ┃ main 1 this 0 -1
1200-1250 ms ┃ main 1 this 0 -1
1600-1650 ms ┃ main 1 this 0 -1
2000-2050 ms ┃ main 1 this 0 -1
2400-2450 ms ┃ main 1 this 0 -1
6050-6100 ms ┃ main 1 this 0 -1
7900-7950 ms ┃ main 0 this 1 +1

ℹ️ Metric definitions & methodology

_{The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.}

_{Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.}

_{Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)}

_{Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost}

_{🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000}

_{All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.}

_{Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.}

github-actions · 2026-08-13T22:30:36Z

Sim World

Simulated world deterministic testing for races. Traces

🟠 Mint-ordered log — 6 fail of 41 total

log=mint-ordered · fence=per-spec

scenario	outcome	events	virt	replay	violations
✅ `smoke-no-steps`	completed	3	0ms	ok	0
✅ `smoke-one-step`	completed	6	0ms	ok	0
✅ `hook-at-step-started`	completed	12	0ms	ok	0
✅ `hook-at-step-completed`	completed	12	0ms	ok	0
✅ `hook-at-hook-created`	completed	12	0ms	ok	0
✅ `deadline-hook-wins`	completed	7	1.0h	ok	0
✅ `deadline-expires`	completed	7	1.0h	ok	0
✅ `long-sleep`	completed	11	30.0d	ok	0
✅ `hook-never-arrives`	stalled	3	0ms	skipped	0
✅ `step-retries-twice`	completed	10	2.0s	ok	0
✅ `parallel-steps`	completed	9	0ms	ok	0
✅ `hook-on-execution-state`	completed	12	0ms	ok	0
✅ `peek-hook-before-branch`	completed	12	0ms	ok	0
✅ `peek-hook-after-branch`	completed	12	0ms	ok	0
✅ `peek-hook-at-registration`	completed	12	0ms	ok	0
✅ `race-hook-before-probe`	completed	12	0ms	ok	0
✅ `race-hook-after-probe`	completed	12	0ms	ok	0
✅ `race-duplicate-delivery`	completed	13	0ms	ok	0
✅ `attr-hook-before-step`	completed	11	0ms	ok	0
✅ `attr-hook-after-step`	completed	11	0ms	ok	0
✅ `attr-from-step-body`	completed	13	0ms	ok	0
✅ `fork-hook-after-timeout`	completed	14	1.0m	ok	0
✅ `fork-hook-before-timeout`	completed	14	1.0m	ok	0
✅ `count-hook-after-timeout`	completed	17	1.0m	ok	0
✅ `count-hook-before-timeout`	completed	20	1.0m	ok	0
❌ `stale-read-step-count-fork`	completed	17	1.0m	MISMATCH	1
❌ `stale-read-equal-step-counts`	completed	14	1.0m	MISMATCH	1
❌ `step-vs-step-fork`	completed	12	0ms	MISMATCH	1
❌ `step-vs-step-fork-fenced`	completed	12	0ms	MISMATCH	1
✅ `fence-catches-benign-direction`	completed	12	5ms	ok	0
❌ `in-flight-before-decision`	completed	17	1.0m	MISMATCH	1
✅ `in-flight-before-decision-counted`	completed	20	1.0m	ok	0
❌ `in-flight-after-decision`	failed	14	2.0m	MISMATCH	1
✅ `stale-read-step-count-fork-fenced`	completed	20	1.0m	ok	0
✅ `fork-hook-wins`	completed	13	1.0m	ok	0
✅ `fork-timeout-wins`	completed	13	1.0m	ok	0
✅ `unclaimed-payload-under-fork`	completed	17	1.0m	ok	0
✅ `claimed-payload-under-fork`	completed	17	1.0m	ok	0
✅ `writers-independent-step-bodies`	completed	12	0ms	ok	0
✅ `writers-scripted-tempo`	completed	12	0ms	ok	0
✅ `cancel-mid-step`	cancelled	7	0ms	skipped	0

Full trace: world-sim-mint.txt

🟢 Append-only log — 0 fail of 41 total

log=append-only · fence=per-spec

scenario	outcome	events	virt	replay
✅ `smoke-no-steps`	completed	3	0ms	ok
✅ `smoke-one-step`	completed	6	0ms	ok
✅ `hook-at-step-started`	completed	12	0ms	ok
✅ `hook-at-step-completed`	completed	12	0ms	ok
✅ `hook-at-hook-created`	completed	12	0ms	ok
✅ `deadline-hook-wins`	completed	7	1.0h	ok
✅ `deadline-expires`	completed	7	1.0h	ok
✅ `long-sleep`	completed	11	30.0d	ok
✅ `hook-never-arrives`	stalled	3	0ms	skipped
✅ `step-retries-twice`	completed	10	2.0s	ok
✅ `parallel-steps`	completed	9	0ms	ok
✅ `hook-on-execution-state`	completed	12	0ms	ok
✅ `peek-hook-before-branch`	completed	12	0ms	ok
✅ `peek-hook-after-branch`	completed	12	0ms	ok
✅ `peek-hook-at-registration`	completed	12	0ms	ok
✅ `race-hook-before-probe`	completed	12	0ms	ok
✅ `race-hook-after-probe`	completed	12	0ms	ok
✅ `race-duplicate-delivery`	completed	13	0ms	ok
✅ `attr-hook-before-step`	completed	11	0ms	ok
✅ `attr-hook-after-step`	completed	11	0ms	ok
✅ `attr-from-step-body`	completed	13	0ms	ok
✅ `fork-hook-after-timeout`	completed	14	1.0m	ok
✅ `fork-hook-before-timeout`	completed	14	1.0m	ok
✅ `count-hook-after-timeout`	completed	17	1.0m	ok
✅ `count-hook-before-timeout`	completed	20	1.0m	ok
✅ `stale-read-step-count-fork`	completed	20	1.0m	ok
✅ `stale-read-equal-step-counts`	completed	14	1.0m	ok
✅ `step-vs-step-fork`	completed	12	0ms	ok
✅ `step-vs-step-fork-fenced`	completed	12	0ms	ok
✅ `fence-catches-benign-direction`	completed	12	5ms	ok
✅ `in-flight-before-decision`	completed	17	1.0m	ok
✅ `in-flight-before-decision-counted`	completed	17	1.0m	ok
✅ `in-flight-after-decision`	completed	19	2.0m	ok
✅ `stale-read-step-count-fork-fenced`	completed	20	1.0m	ok
✅ `fork-hook-wins`	completed	13	1.0m	ok
✅ `fork-timeout-wins`	completed	13	1.0m	ok
✅ `unclaimed-payload-under-fork`	completed	17	1.0m	ok
✅ `claimed-payload-under-fork`	completed	17	1.0m	ok
✅ `writers-independent-step-bodies`	completed	12	0ms	ok
✅ `writers-scripted-tempo`	completed	12	0ms	ok
✅ `cancel-mid-step`	cancelled	7	0ms	skipped

Full trace: world-sim-append-only.txt

VaguelySerious

The flaky annotation addition is great. We should still take care to check the flakes, and hopefully we'll get some more time soon to fix the actual tests, but this is great in the meantime

github-actions · 2026-08-14T17:12:20Z

Backport to stable failed for c041d3d due to a workflow error (backport job run).

This is usually an infrastructure problem (e.g. the configured AI model could not be found, an AI Gateway error, or an opencode crash) rather than a merge conflict. Check the job logs linked above for details.

Once the underlying issue is fixed, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

c041d3d231f8a75236311df56a68bd4ca104be22

## Summary & Motivation The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for `nextjs-turbopack` and `nextjs-webpack`, × node/quickjs) were pinned to `16.3.0-canary.2`. This bumps the pin to the current canary, `16.3.1-canary.17`, in all six spots in `tests.yml`. Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h `minimumReleaseAge` gate, so the "Setup canary" `pnpm install --no-frozen-lockfile` fails with `ERR_PNPM_NO_MATURE_MATCHING_VERSION` (reproduced locally with this exact version). This adds `next` and its lockstep-published `@next/*` companion packages (`@next/env`, `@next/swc-*`) to `minimumReleaseAgeExclude` — both are Vercel-published, matching the trust model of the existing exclusions (`@vercel/*`, `turbo`, `esbuild`). ## Validation The full e2e suite was run locally against `next@16.3.1-canary.17` on all three worlds (staged tarball workbenches, same as CI's `prepare-workbench-path`): | World | App / mode | Result | | --- | --- | --- | | `world-local` | nextjs-webpack, dev server, node vm | `dev.test.ts` 5 passed / 4 skipped (incl. the HMR fuzz test) · `test:e2e` 137 passed / 19 skipped | | `world-postgres` | nextjs-turbopack, prod build, node vm | `local-build.test.ts` 13 passed · `test:e2e` 137 passed / 19 skipped¹ | | `world-vercel` | nextjs-turbopack, preview deployment (`dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ`), node vm | `test:e2e` 134 passed / 22 skipped | ¹ First pass failed the 3 `pages router` tests because the local harness ran `local-build.test.ts` without `CI=true`, which deletes the `workflow-sourcemap-warning-fixture` package the built output still references (the test preserves it only when `CI=true`, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue. Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got. ## Context on current `main` CI Recent `main` runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed `main` run (with #3530's e2e retry) passed every canary lane and failed only the *stable* webpack dev lane. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

alangenfeld added 2 commits August 13, 2026 17:24

Add changeset
b40e1aa
Signed-off-by: Alex Langenfeld <alex.langenfeld@vercel.com>

vercelBotdeployed to Preview – workflow-web August 13, 2026 22:26 View deployment

vercelBotdeployed to Preview – workbench-nestjs-workflow August 13, 2026 22:26 View deployment

vercelBotdeployed to Preview – workbench-express-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-fastify-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-hono-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-nitro-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-nuxt-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workflow-tarballs August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-astro-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-vite-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – example-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-tanstack-start-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – workbench-sveltekit-workflow August 13, 2026 22:27 View deployment

vercelBotdeployed to Preview – example-nextjs-workflow-turbopack August 13, 2026 22:28 View deployment

vercelBotdeployed to Preview – example-nextjs-workflow-webpack August 13, 2026 22:28 View deployment

vercelBotdeployed to Preview – workflow-docs August 13, 2026 22:28 View deployment

vercelBotdeployed to Preview – workflow-swc-playground August 13, 2026 22:30 View deployment

alangenfeld marked this pull request as ready for review August 14, 2026 14:47

alangenfeld requested a review from a team as a code owner August 14, 2026 14:47

VaguelySerious approved these changes Aug 14, 2026
View reviewed changes

alangenfeld merged commit c041d3d into mainAug 14, 2026
293 of 297 checks passed

alangenfeld deleted the alangenfeld/e2e-ci-retry branch August 14, 2026 17:08

github-actionsBot mentioned this pull request Aug 14, 2026
Version Packages (beta) #3466
Merged

alangenfeld mentioned this pull request Aug 14, 2026
[stable] Backport: Retry e2e tests once in CI, keeping retried tests visible (#3530) #3555
Closed

pranaygp mentioned this pull request Aug 14, 2026
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 #3559
Merged

Navigation Menu

Uh oh!

[ci] Retry e2e tests once in CI, keeping retried tests visible - #3530

[ci] Retry e2e tests once in CI, keeping retried tests visible#3530
alangenfeld merged 2 commits into
mainfrom
alangenfeld/e2e-ci-retry

alangenfeld commented Aug 13, 2026

Uh oh!

changeset-botBot commented Aug 13, 2026

Uh oh!

vercelBot commented Aug 13, 2026•
edited
Loading

Uh oh!

github-actionsBot commented Aug 13, 2026•
edited
Loading

Uh oh!

github-actionsBot commented Aug 13, 2026•
edited
Loading

Uh oh!

github-actionsBot commented Aug 13, 2026

Uh oh!

VaguelySerious left a comment

Uh oh!

Uh oh!

github-actionsBot commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

alangenfeld commented Aug 13, 2026

Summary & Motivation

Test Plan

Uh oh!

changeset-botBot commented Aug 13, 2026

🦋 Changeset detected

Uh oh!

vercelBot commented Aug 13, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actionsBot commented Aug 13, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 E2E Test Results

⚠️ Flaky E2E Tests (passed on retry)

E2E Test Summary

Uh oh!

github-actionsBot commented Aug 13, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Workflow Benchmarks

Uh oh!

github-actionsBot commented Aug 13, 2026

Sim World

Uh oh!

VaguelySerious left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actionsBot commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercelBot commented Aug 13, 2026•
edited
Loading

github-actionsBot commented Aug 13, 2026•
edited
Loading

github-actionsBot commented Aug 13, 2026•
edited
Loading