Skip to content

[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559

Merged
pranaygp merged 1 commit into
mainfrom
pgp/bump-next-canary-pin
Aug 17, 2026
Merged

[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom
pgp/bump-next-canary-pin

Conversation

@pranaygp

@pranaygppranaygp commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Summary & Motivation

The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.

Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).

Validation

The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):

WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped

¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.

Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.

Context on current main CI

Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.

🤖 Generated with Claude Code

@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI lite review requested due to automatic review settings August 14, 2026 18:30
@pranaygp
pranaygp requested a review from a team as a code ownerAugust 14, 2026 18:30
@changeset-bot

changeset-botBot commented Aug 14, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: de1d17f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel

vercelBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm

@github-actions

github-actionsBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

✅ All tests passed

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

  • webhookWorkflow (tanstack-start)

E2E Test Summary

Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category

✅ ▲ Vercel Production

AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028

✅ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 14, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)

 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1
ℹ️ Metric definitions & methodology

The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.

CopilotAI reviewed Aug 14, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.

Changes:

  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actionsBot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 Mint-ordered log — 3 fail of 41 total

log=mint-ordered · fence=per-spec

scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0

Full trace: world-sim-mint.txt

🟢 Append-only log — 0 fail of 41 total

log=append-only · fence=per-spec

scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0

Full trace: world-sim-append-only.txt

@pranaygp
pranaygp merged commit 64b8866 into mainAug 17, 2026
168 of 171 checks passed
@pranaygp
pranaygp deleted the pgp/bump-next-canary-pin branch August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 by pranaygp · Pull Request #3559 · vercel/workflow · GitHub
Skip to content
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Merged
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Aug 17, 2026
Merged
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Conversation
@pranaygp
@pranaygppranaygp
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.
Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).
Validation
The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):
WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped
¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.
Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current main CI
Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.
🤖 Generated with Claude Code
@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI
lite review requested due to automatic review settings
August 14, 2026 18:30
@pranaygp
pranaygp
requested a review
from a team
as a code ownerAugust 14, 2026 18:30
@changeset-bot
changeset-botBot
commented
Aug 14, 2026

Copy link


Copy Markdown

⚠️ No Changeset found
Latest commit: de1d17f
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • webhookWorkflow (tanstack-start)
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)
 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1

ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
CopilotAI
reviewed
Aug 14, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
@github-actions
github-actionsBot
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
@pranaygp
pranaygp
merged commit 64b8866
into
mainAug 17, 2026

168 of 171 checks passed

@pranaygp
pranaygp
deleted the

pgp/bump-next-canary-pin

branch
August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 by pranaygp · Pull Request #3559 · vercel/workflow · GitHub
Skip to content
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Merged
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Aug 17, 2026
Merged
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Conversation
@pranaygp
@pranaygppranaygp
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.
Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).
Validation
The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):
WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped
¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.
Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current main CI
Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.
🤖 Generated with Claude Code
@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI
lite review requested due to automatic review settings
August 14, 2026 18:30
@pranaygp
pranaygp
requested a review
from a team
as a code ownerAugust 14, 2026 18:30
@changeset-bot
changeset-botBot
commented
Aug 14, 2026

Copy link


Copy Markdown

⚠️ No Changeset found
Latest commit: de1d17f
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • webhookWorkflow (tanstack-start)
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)
 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1

ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
CopilotAI
reviewed
Aug 14, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
@github-actions
github-actionsBot
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
@pranaygp
pranaygp
merged commit 64b8866
into
mainAug 17, 2026

168 of 171 checks passed

@pranaygp
pranaygp
deleted the

pgp/bump-next-canary-pin

branch
August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 by pranaygp · Pull Request #3559 · vercel/workflow · GitHub
Skip to content
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Merged
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Aug 17, 2026
Merged
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Conversation
@pranaygp
@pranaygppranaygp
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.
Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).
Validation
The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):
WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped
¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.
Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current main CI
Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.
🤖 Generated with Claude Code
@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI
lite review requested due to automatic review settings
August 14, 2026 18:30
@pranaygp
pranaygp
requested a review
from a team
as a code ownerAugust 14, 2026 18:30
@changeset-bot
changeset-botBot
commented
Aug 14, 2026

Copy link


Copy Markdown

⚠️ No Changeset found
Latest commit: de1d17f
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • webhookWorkflow (tanstack-start)
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)
 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1

ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
CopilotAI
reviewed
Aug 14, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
@github-actions
github-actionsBot
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
@pranaygp
pranaygp
merged commit 64b8866
into
mainAug 17, 2026

168 of 171 checks passed

@pranaygp
pranaygp
deleted the

pgp/bump-next-canary-pin

branch
August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 by pranaygp · Pull Request #3559 · vercel/workflow · GitHub
Skip to content
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Merged
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Aug 17, 2026
Merged
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Conversation
@pranaygp
@pranaygppranaygp
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.
Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).
Validation
The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):
WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped
¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.
Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current main CI
Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.
🤖 Generated with Claude Code
@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI
lite review requested due to automatic review settings
August 14, 2026 18:30
@pranaygp
pranaygp
requested a review
from a team
as a code ownerAugust 14, 2026 18:30
@changeset-bot
changeset-botBot
commented
Aug 14, 2026

Copy link


Copy Markdown

⚠️ No Changeset found
Latest commit: de1d17f
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • webhookWorkflow (tanstack-start)
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)
 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1

ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
CopilotAI
reviewed
Aug 14, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
@github-actions
github-actionsBot
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
@pranaygp
pranaygp
merged commit 64b8866
into
mainAug 17, 2026

168 of 171 checks passed

@pranaygp
pranaygp
deleted the

pgp/bump-next-canary-pin

branch
August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 by pranaygp · Pull Request #3559 · vercel/workflow · GitHub
Skip to content
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Merged
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Aug 17, 2026
Merged
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Conversation
@pranaygp
@pranaygppranaygp
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.
Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).
Validation
The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):
WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped
¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.
Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current main CI
Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.
🤖 Generated with Claude Code
@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI
lite review requested due to automatic review settings
August 14, 2026 18:30
@pranaygp
pranaygp
requested a review
from a team
as a code ownerAugust 14, 2026 18:30
@changeset-bot
changeset-botBot
commented
Aug 14, 2026

Copy link


Copy Markdown

⚠️ No Changeset found
Latest commit: de1d17f
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • webhookWorkflow (tanstack-start)
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)
 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1

ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
CopilotAI
reviewed
Aug 14, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
@github-actions
github-actionsBot
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
@pranaygp
pranaygp
merged commit 64b8866
into
mainAug 17, 2026

168 of 171 checks passed

@pranaygp
pranaygp
deleted the

pgp/bump-next-canary-pin

branch
August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
})();
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 by pranaygp · Pull Request #3559 · vercel/workflow · GitHub
Skip to content
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17 - #3559
Merged
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Aug 17, 2026
Merged
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17#3559
pranaygp merged 1 commit into
mainfrom 
pgp/bump-next-canary-pin
Conversation
@pranaygp
@pranaygppranaygp
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Summary & Motivation
The canary e2e lanes (E2E Local Dev / Local Prod / Local Postgres for nextjs-turbopack and nextjs-webpack, × node/quickjs) were pinned to 16.3.0-canary.2. This bumps the pin to the current canary, 16.3.1-canary.17, in all six spots in tests.yml.
Bumping the pin alone is not enough: fresh Next canaries are younger than the repo's 48h minimumReleaseAge gate, so the "Setup canary" pnpm install --no-frozen-lockfile fails with ERR_PNPM_NO_MATURE_MATCHING_VERSION (reproduced locally with this exact version). This adds next and its lockstep-published @next/* companion packages (@next/env, @next/swc-*) to minimumReleaseAgeExclude — both are Vercel-published, matching the trust model of the existing exclusions (@vercel/*, turbo, esbuild).
Validation
The full e2e suite was run locally against next@16.3.1-canary.17 on all three worlds (staged tarball workbenches, same as CI's prepare-workbench-path):
WorldApp / modeResult
world-localnextjs-webpack, dev server, node vmdev.test.ts 5 passed / 4 skipped (incl. the HMR fuzz test) · test:e2e 137 passed / 19 skipped
world-postgresnextjs-turbopack, prod build, node vmlocal-build.test.ts 13 passed · test:e2e 137 passed / 19 skipped¹
world-vercelnextjs-turbopack, preview deployment (dpl_3siALD2WyrsrdPFjmRY3ymwEHZiJ), node vmtest:e2e 134 passed / 22 skipped
¹ First pass failed the 3 pages router tests because the local harness ran local-build.test.ts without CI=true, which deletes the workflow-sourcemap-warning-fixture package the built output still references (the test preserves it only when CI=true, exactly as its comment warns). With the fixture preserved, all 3 pass. Not a canary issue.
Note: the vercel world has no canary lane in CI (deployments build from the committed stable pin), so the preview-deployment run above is the only canary coverage it got.
Context on current main CI
Recent main runs are red, but not because of the canary pin: the dominant failure is the nextjs-webpack local-dev HMR fuzz race (SDK-side, fix in flight in #3529), which hits the stable and canary lanes alike; the rest are the known rotating vercel-prod flakes. The latest completed main run (with #3530's e2e retry) passed every canary lane and failed only the stable webpack dev lane.
🤖 Generated with Claude Code
@pranaygp@claude
[ci] Bump Next.js canary e2e pin to 16.3.1-canary.17
de1d17f
The canary e2e lanes (Local Dev / Local Prod / Local Postgres for
nextjs-turbopack and nextjs-webpack) were pinned to 16.3.0-canary.2.
Bump the pin to the current canary, 16.3.1-canary.17.
Fresh canaries are younger than the repo's 48h minimumReleaseAge gate,
which rejects the pinned version at the "Setup canary" install step, so
exclude next and its lockstep-published @next/* companion packages from
the gate — both are Vercel-published, matching the existing exclusions
(@vercel/*, turbo, esbuild).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CopilotAI
lite review requested due to automatic review settings
August 14, 2026 18:30
@pranaygp
pranaygp
requested a review
from a team
as a code ownerAugust 14, 2026 18:30
@changeset-bot
changeset-botBot
commented
Aug 14, 2026

Copy link


Copy Markdown

⚠️ No Changeset found
Latest commit: de1d17f
Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.
This PR includes no changesets
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Click here to learn what changesets are, and how to add one.
Click here if you're a maintainer who wants to add a changeset to this PR
@vercel
vercelBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 14, 2026 6:36pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 14, 2026 6:36pm
example-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-astro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-express-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-fastify-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-hono-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nestjs-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nitro-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-nuxt-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-python-workflowErrorErrorAug 14, 2026 6:36pm
workbench-sveltekit-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workbench-vite-workflowReadyReadyPreviewAug 14, 2026 6:36pm
workflow-docsReadyReadyPreview, v0Aug 14, 2026 6:36pm
workflow-swc-playgroundReadyReadyPreviewAug 14, 2026 6:36pm
workflow-tarballsReadyReadyPreviewAug 14, 2026 6:36pm
workflow-webReadyReadyPreviewAug 14, 2026 6:36pm
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
⚠️ Flaky E2E Tests (passed on retry)
These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.
  • webhookWorkflow (tanstack-start)
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production346605904056
✅ 💻 Local Development381005584368
✅ 📦 Local Production381005584368
✅ 🐘 Local Postgres381005584368
✅ 🪟 Windows31200312
✅ vercel-multi-region270027
Total152350226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nest-node128028
✅ nest-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ tanstack-start-node128028
✅ tanstack-start-quickjs128028
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nest-stable-node130026
✅ nest-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ tanstack-start-node130026
✅ tanstack-start-quickjs130026
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@github-actions
github-actionsBot
commented
Aug 14, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit de1d17f · Fri, 14 Aug 2026 18:53:33 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep475 (-56%) 💚1473 🔴 (+2.4%)1487 🔴 (-1.5%)1710 🔴 (+5.4%)30
TTFSstream1411 (+175%) 🔻1467 🔴 (+13%)1479 🔴 (+13%)1505 🔴 (+6.6%)30
TTFShook + stream545 (-12%)1794 🔴 (+8.9%)1907 🔴 (+8.0%)1955 🔴 (+0.5%)30
Fan-out TTFSPromise.all(100 steps)8913 (±0%)10485 (+4.5%)10604 (+4.2%)10705 (+3.9%)10
Fan-out TTLSPromise.all(100 steps)17535 (+0.7%)19203 (+1.6%)19356 (+2.0%)20686 (+2.8%)10
STSO1020 steps (inline)132 (-5.7%)173 (-25%) 💚195 (-30%) 💚328 (-56%) 💚1019
WO1020 steps172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚172912 (-24%) 💚1
SLstream latency87 (-26%) 💚124 🔴 (-32%) 💚158 🔴 (-34%) 💚444 🔴 (-36%) 💚30
SOstream overhead (text)94 (-31%) 💚177 (-25%) 💚198 (-34%) 💚249 (-56%) 💚30
SOstream overhead (structured)109 (-26%) 💚176 (-33%) 💚216 (-26%) 💚3255 🔴 (+686%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 226900ms → this run 172717ms (Δ -54183ms, -24%)
 100-150 ms █░░░░░┃ main 11 this 204 +193
150-200 ms ████████████████░░░░░░░┃ main 499 this 730 +231
200-250 ms █┃█████████ main 338 this 51 -287
250-300 ms ┃██ main 90 this 16 -74
300-350 ms ┃ main 27 this 10 -17
350-400 ms ┃ main 20 this 2 -18
400-450 ms ┃ main 5 this 2 -3
450-500 ms ┃ main 4 this 1 -3
500-550 ms ┃ main 3 this 0 -3
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 1 this 1 +0
650-700 ms ┃ main 5 this 0 -5
700-750 ms ┃ main 8 this 1 -7
800-850 ms ┃ main 2 this 0 -2
850-900 ms ┃ main 2 this 0 -2
900-950 ms ┃ main 2 this 0 -2
1050-1100 ms ┃ main 1 this 0 -1
1150-1200 ms ┃ main 1 this 0 -1

ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. Fan-out TTFS/TTLS are the first and last step completions of a single Promise.all over trivial steps, from the same anchor, so the gap between the two rows is the spread the runtime adds across the fan-out. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
CopilotAI
reviewed
Aug 14, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Updates the CI “Next.js canary” E2E lanes to a newer Next.js canary while ensuring pnpm’s minimumReleaseAge gate doesn’t block installs of freshly published canary artifacts.
Changes:
  • Bump the canary pin used by the local E2E jobs from 16.3.0-canary.2 to 16.3.1-canary.17.
  • Add next and @next/* to pnpm’s minimumReleaseAgeExclude so CI can install new canaries without ERR_PNPM_NO_MATURE_MATCHING_VERSION.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
FileDescription
pnpm-workspace.yamlExcludes next / @next/* from the minimum release-age gate to allow fresh canary installs in CI.
.github/workflows/tests.ymlUpdates the hardcoded canary pin used in the canary setup step across the local E2E jobs.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
@github-actions
github-actionsBot
commented
Aug 14, 2026

Copy link


Copy Markdown

Contributor
Sim World
Simulated world deterministic testing for races. Traces
🟠 Mint-ordered log — 3 fail of 41 total
log=mint-ordered · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
❌ in-flight-before-decisionfailed91.0mMISMATCH1
❌ in-flight-before-decision-countedfailed91.0mMISMATCH1
❌ in-flight-after-decisionfailed91.0mMISMATCH1
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-mint.txt
🟢 Append-only log — 0 fail of 41 total
log=append-only · fence=per-spec
scenariooutcomeeventsvirtreplayviolations
✅ smoke-no-stepscompleted30msok0
✅ smoke-one-stepcompleted60msok0
✅ hook-at-step-startedcompleted120msok0
✅ hook-at-step-completedcompleted120msok0
✅ hook-at-hook-createdcompleted120msok0
✅ deadline-hook-winscompleted71.0hok0
✅ deadline-expirescompleted71.0hok0
✅ long-sleepcompleted1130.0dok0
✅ hook-never-arrivesstalled30msskipped0
✅ step-retries-twicecompleted102.0sok0
✅ parallel-stepscompleted90msok0
✅ hook-on-execution-statecompleted120msok0
✅ peek-hook-before-branchcompleted120msok0
✅ peek-hook-after-branchcompleted120msok0
✅ peek-hook-at-registrationcompleted120msok0
✅ race-hook-before-probecompleted120msok0
✅ race-hook-after-probecompleted120msok0
✅ race-duplicate-deliverycompleted130msok0
✅ attr-hook-before-stepcompleted110msok0
✅ attr-hook-after-stepcompleted110msok0
✅ attr-from-step-bodycompleted130msok0
✅ fork-hook-after-timeoutcompleted141.0mok0
✅ fork-hook-before-timeoutcompleted141.0mok0
✅ count-hook-after-timeoutcompleted171.0mok0
✅ count-hook-before-timeoutcompleted201.0mok0
✅ stale-read-step-count-forkcompleted201.0mok0
✅ stale-read-equal-step-countscompleted141.0mok0
✅ step-vs-step-forkcompleted120msok0
✅ step-vs-step-fork-fencedcompleted120msok0
✅ fence-catches-benign-directioncompleted125msok0
✅ in-flight-before-decisioncompleted171.0mok0
✅ in-flight-before-decision-countedcompleted171.0mok0
✅ in-flight-after-decisioncompleted192.0mok0
✅ stale-read-step-count-fork-fencedcompleted201.0mok0
✅ fork-hook-winscompleted131.0mok0
✅ fork-timeout-winscompleted131.0mok0
✅ unclaimed-payload-under-forkcompleted171.0mok0
✅ claimed-payload-under-forkcompleted171.0mok0
✅ writers-independent-step-bodiescompleted120msok0
✅ writers-scripted-tempocompleted120msok0
✅ cancel-mid-stepcancelled70msskipped0
Full trace: world-sim-append-only.txt
@pranaygp
pranaygp
merged commit 64b8866
into
mainAug 17, 2026

168 of 171 checks passed

@pranaygp
pranaygp
deleted the

pgp/bump-next-canary-pin

branch
August 17, 2026 17:37
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@TooTallNateTooTallNateTooTallNate approved these changes
@VaguelySeriousVaguelySeriousVaguelySerious approved these changes

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious