Skip to content

ci: stop deploying changeset-release/main, run its e2e against production - #3243

Merged
pranaygp merged 2 commits into
mainfrom
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged

ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom
pgp/skip-changeset-release-deploys

Conversation

@pranaygp

@pranaygppranaygp commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

The incident

The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).

The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.

You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:

Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main

Every Vercel project rooted in this repo now sets, in its vercel.json:

{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }

Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:

ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)

Notes:

  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.

Part 2 — the changeset PR's e2e runs against production

With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.

In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:

  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.

No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).

The resolution logic

.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.

Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:

  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.

It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.

Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:

deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung

Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:

  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.

event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.

Notes for review

  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.

🤖 Generated with Claude Code

@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI review requested due to automatic review settings July 30, 2026 23:36
@pranaygp
pranaygp requested review from a team and ijjk as code ownersJuly 30, 2026 23:36
@changeset-bot

changeset-botBot commented Jul 30, 2026•
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5daef6a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 30, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm

@github-actions

github-actionsBot commented Jul 30, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)

 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)

Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)

2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1
📜 Previous results (1)

960fef2

Thu, 30 Jul 2026 23:59:06 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology

The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.

@github-actions

github-actionsBot commented Jul 30, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

❌ Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (1 failed)

sveltekit (1 failed):

  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028

✅ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack15400

✅ 📋 Other

AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

CopilotAI reviewed Jul 30, 2026
View reviewed changes

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.

Changes:

  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread.github/scripts/resolve-production-deployment.mjs Outdated
Comment on lines +58 to +60
const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;

@pranaygppranaygpJul 31, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.

TooTallNate
TooTallNate approved these changes Jul 31, 2026
View reviewed changes

@TooTallNateTooTallNate left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.

On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.

No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.

Comment thread.github/scripts/resolve-production-deployment.mjs Outdated
const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);

@TooTallNateTooTallNateJul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.

No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.

@pranaygppranaygpJul 31, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.

@TooTallNateTooTallNate mentioned this pull request Jul 31, 2026
Open
TooTallNate
TooTallNate reviewed Jul 31, 2026
View reviewed changes

@TooTallNateTooTallNate left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.

Comment thread.github/workflows/tests.yml Outdated
id: prodDeployment
if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
env:
VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}

@TooTallNateTooTallNateJul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:

- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}

The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.

Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.

@pranaygppranaygpJul 31, 2026

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.

@TooTallNateTooTallNate mentioned this pull request Jul 31, 2026
Merged
@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp merged commit 11dc036 into mainJul 31, 2026
105 of 107 checks passed
@pranaygp
pranaygp deleted the pgp/skip-changeset-release-deploys branch July 31, 2026 17:09
@github-actions

github-actionsBot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

No backport to stable for 11dc036 (AI decision).

This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit that referenced this pull request Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
ci: stop deploying changeset-release/main, run its e2e against production by pranaygp · Pull Request #3243 · vercel/workflow · GitHub
Skip to content
ci: stop deploying changeset-release/main, run its e2e against production - #3243
Merged
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Jul 31, 2026
Merged
ci: stop deploying changeset-release/main, run its e2e against production#3243
pranaygp merged 2 commits into
mainfrom 
pgp/skip-changeset-release-deploys
Conversation
@pranaygp
@pranaygppranaygp
commented
Jul 30, 2026

Copy link


Copy Markdown

Contributor
The incident
The changesets action force-pushes changeset-release/main, and that branch can point at exactly main's HEAD SHA. When it does, Vercel builds the same commit twice per project: a production deployment (from main) and a preview deployment (from changeset-release/main).
The Vercel GitHub integration keeps one commit status per project per SHA, overwritten by whichever deployment finishes last. On 2026-07-30 the preview finished last, poisoning the status for main's HEAD. vercel/wait-for-deployment-action reads the deployment ID out of that commit status, so a job asking for environment: production got handed a preview deployment ID — and the e2e/repro runs forked across production and preview environments.
You can still see both statuses on main's current HEAD (32ac8e73f), pointing at two different deployments of one commit:
Vercel – example-nextjs-workflow-turbopack success .../4zdDfSiqwLVcfqd5YWdsSzLaGrfb
Vercel – example-nextjs-workflow-turbopack success .../27E2xjsdV3HCDGBnJN31e7uDDykF

Part 1 — stop deploying changeset-release/main
Every Vercel project rooted in this repo now sets, in its vercel.json:
{ "git": { "deploymentEnabled": { "changeset-release/main": false } } }
Applied to all 17 project roots, verified against the Vercel API (rootDirectory of each project linked to vercel/workflow) rather than guessed from the directory tree:
ProjectRoot
example-workflowworkbench/example
example-nextjs-workflow-turbopackworkbench/nextjs-turbopack
example-nextjs-workflow-webpackworkbench/nextjs-webpack
workbench-nitro-workflowworkbench/nitro-v3
workbench-vite-workflowworkbench/vite
workbench-nuxt-workflowworkbench/nuxt
workbench-sveltekit-workflowworkbench/sveltekit
workbench-hono-workflowworkbench/hono
workbench-express-workflowworkbench/express
workbench-fastify-workflowworkbench/fastify
workbench-nestjs-workflowworkbench/nest
workbench-astro-workflowworkbench/astro
workbench-tanstack-start-workflowworkbench/tanstack-start
workflow-swc-playgroundworkbench/swc-playground
workflow-tarballstarballs
workflow-docsdocs (new vercel.json)
workflow-webpackages/web (new vercel.json)
Notes:
  • docs/ and packages/web/ had no vercel.json. The new files contain only $schema + the git key, so no project setting is overridden (both projects have all build settings on auto-detect, and git.deploymentEnabled was unset). packages/web/vercel.json is not in @workflow/web's files allowlist, so it is not published.
  • workbench/nitro-v2/vercel.json gets the key too, for uniformity — no project currently deploys from it (workbench/nitro is a symlink to nitro-v3, which is the real root).
  • All 17 projects had git.deploymentEnabled unset at the project level, so nothing conflicts with the new file-based config.
Part 2 — the changeset PR's e2e runs against production
With no deployments of its own, the changeset PR's deployment waits would hang. Its content is main plus a version-bump commit, so its coverage now comes from the production deployment main already produced.
In tests.yml, both deployment-waiting jobs (e2e-vercel-prod and e2e-vercel-multi-region) gain a branch:
  • The existing waitForDeployment step is gated on !startsWith(github.head_ref, 'changeset-release/') — main-push and normal-PR behavior is untouched.
  • A new step runs node .github/scripts/resolve-production-deployment.mjs for the changeset case, emitting deployment-url / deployment-id / deployment-state in exactly the wait action's format. Downstream env reads steps.waitForDeployment.outputs.X || steps.prodDeployment.outputs.X, so the wiring is otherwise unchanged.
  • WORKFLOW_VERCEL_ENV becomes production for the changeset case. This is load-bearing, not cosmetic: the harness builds its Vercel world with environment: WORKFLOW_VERCEL_ENV (packages/core/e2e/utils.ts), so leaving it preview would have it looking for runs in the preview environment while the deployment writes to production.
  • VERCEL_WORKFLOW_SERVER_URL is now also unset for the changeset case, matching main. A production deployment is wired to the production workflow-server, and the harness has to read run state from the same server the app writes it to.
No permissions: changes were needed — both jobs already carry id-token: write for the trusted-sources OIDC bypass, which the production path relies on (all these projects have Vercel Authentication on all deployments).
The resolution logic
.github/scripts/resolve-production-deployment.mjs polls GET /v6/deployments?app=<slug>&target=production&limit=100 and looks for a deployment whose meta.githubCommitSha equals github.event.pull_request.base.sha — the base SHA, because that is the main commit whose production deployment carries the code under test.
Keying on the SHA rather than "whatever is currently aliased to production" is what makes the result correct rather than merely recent:
  • READY match → emit its URL/ID and exit 0.
  • Match still building (QUEUED / BUILDING / …) → keep polling; the changeset PR's event fires while main's production deploy is often still in flight.
  • Match in a terminal failure state (ERROR / CANCELED / DELETED) → fail immediately with the deployment listed, instead of waiting out the clock on a build that will never be ready.
  • No match → keep polling, then time out (1000s, matching the wait action) printing the ten most recent production deployments with state + SHA + timestamp, so the log alone shows whether the build failed or the base SHA was unexpected.
  • Transient API errors → logged and retried until the deadline, never fatal on their own.
It never falls back to a newer or older deployment: testing code that isn't the code under test is worse than failing loudly. limit=100 covers roughly two weeks of main pushes (verified: 100 rows reach back to 2026-07-14), so neither pushes landing after the PR event nor a re-run of an older workflow run can bury the target.
Verified against the live API for several projects. For main's HEAD the script returns byte-identical values to what the wait action produced for the same commit:
deployment-url=https://example-nextjs-workflow-turbopack-kxjvlmjuk.labs.vercel.dev
deployment-id=dpl_4zdDfSiqwLVcfqd5YWdsSzLaGrfb
deployment-state=success

Also included: three other jobs that would have hung
Part 1 removes the deployments that three other workflows wait on, so they needed handling or they would have burned their full timeouts on every changeset PR:
  • docs-checks.yml → docs-preview-smoke skipped. Its docs content is identical to main's, which this same job already checked against production on the push that produced the bump. docs-typecheck still runs.
  • tarballs-checks.yml → tarballs-preview-smoke skipped. The version-bumped tarballs get this same check on the push to main right after the version PR merges, before any consumer sees them.
  • benchmarks.yml → all three PR jobs skipped. Benchmarking would be meaningless here anyway: the code is main's, so it would only compare main's baseline against itself.
event-log-race-repro.yml is untouched (label-gated; changeset PRs are never labeled), and dispatch-front-workflow-release-pr.yml is fully disabled already.
Notes for review
  • AGENTS.md records the invariant, because it is not discoverable from any one file: a new Vercel project needs the git key in its vercel.json, and anything new that waits on a deployment needs the changeset-release branch.
  • The changeset is empty (---\n---\n), matching how CI-only PRs are handled here (e.g. Fix Biome lint violations and add Biome CI check #3222). No published package changes; packages/web/vercel.json sits inside @workflow/web but is excluded from its files.
  • Full verification of the changeset-release path can only happen on the next real "Version Packages" PR, since github.head_ref on this PR is pgp/skip-changeset-release-deploys. This PR's own CI exercises the unchanged normal-PR path, which is the regression risk worth checking here.
🤖 Generated with Claude Code
@pranaygp
ci: stop deploying changeset-release/main, run its e2e against produc…
960fef2
…tion
The changesets action force-pushes `changeset-release/main`, and it can
point at exactly main's HEAD SHA. Vercel keeps one commit status per
project per SHA, so when both a production deployment (from main) and a
preview deployment (from changeset-release/main) are built for the same
commit, whichever finishes last owns the status. On 2026-07-30 the
preview finished last, so `vercel/wait-for-deployment-action` — which
reads the deployment ID out of that status — handed production e2e runs
a preview deployment ID and forked runs across environments.
Disable git deployments for that branch in every Vercel project rooted
in this repo, and give the changeset PR's Vercel e2e lanes a deployment
to test that actually exists: main's production deployment for the PR's
base SHA, resolved by SHA so a mid-flight production build is waited out
rather than silently replaced by an older one.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
CopilotAI
review requested due to automatic review settings
July 30, 2026 23:36
@pranaygp
pranaygp
requested review from
a team and ijjk
as code ownersJuly 30, 2026 23:36
@changeset-bot
changeset-botBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 5daef6a
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewJul 31, 2026 4:39pm
example-nextjs-workflow-webpackReadyReadyPreviewJul 31, 2026 4:39pm
example-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-astro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-express-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-fastify-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-hono-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nestjs-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nitro-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-nuxt-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-sveltekit-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-tanstack-start-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workbench-vite-workflowReadyReadyPreviewJul 31, 2026 4:39pm
workflow-docsReadyReadyPreview, v0Jul 31, 2026 4:39pm
workflow-swc-playgroundReadyReadyPreviewJul 31, 2026 4:39pm
workflow-tarballsReadyReadyPreviewJul 31, 2026 4:39pm
workflow-webReadyReadyPreviewJul 31, 2026 4:39pm
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit 5daef6a · Fri, 31 Jul 2026 16:58:16 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1294 (+147%) 🔻1382 🔴 (+33%) 🔻1435 🔴 (+34%) 🔻1615 🔴 (±0%)30
TTFSstream1294 (+37%) 🔻1432 🔴 (+43%) 🔻1457 🔴 (+42%) 🔻1528 🔴 (+19%) 🔻30
TTFShook + stream1240 (+195%) 🔻1781 🔴 (+34%) 🔻1841 🔴 (+36%) 🔻2220 🔴 (+25%) 🔻30
STSO1020 steps (inline)165 (+1.2%)474 (±0%)538 (+1.3%)735 (+9.5%)1016
STSO1020 steps (queue-hop)2465 (+18%) 🔻3558 (+11%)3558 (+11%)3558 (+11%)3
WO1020 steps409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)409520 (+1.2%)1
SLstream latency110 (+34%) 🔻169 🔴 (+28%) 🔻237 🔴 (+61%) 🔻3831 🔴 (+1682%) 🔻30
SOstream overhead (text)111 (+9.9%)212 (+23%) 🔻251 (+17%) 🔻3479 🔴 (+1292%) 🔻30
SOstream overhead (structured)111 (+0.9%)205 (+21%) 🔻243 (+28%) 🔻273 (+20%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 395612ms → this run 398897ms (Δ +3285ms, +1%)
 150-200 ms ██┃███ main 36 this 21 -15
200-250 ms █████████████┃ main 92 this 90 -2
250-300 ms █████████████████████┃█ main 145 this 138 -7
300-350 ms ██████████████████████░┃ main 139 this 154 +15
350-400 ms ██████████████████████┃ main 138 this 146 +8
400-450 ms ████████████████████┃ main 134 this 136 +2
450-500 ms ██████████████████████┃ main 150 this 149 -1
500-550 ms ████████████████┃ main 107 this 107 +0
550-600 ms ██████┃ main 47 this 42 -5
600-650 ms █┃ main 16 this 10 -6
650-700 ms ┃ main 3 this 9 +6
700-750 ms ┃ main 3 this 5 +2
750-800 ms ┃ main 3 this 4 +1
800-850 ms ┃ main 1 this 0 -1
850-900 ms ┃ main 0 this 1 +1
900-950 ms ┃ main 0 this 2 +2
1000-1050 ms ┃ main 0 this 1 +1
1050-1100 ms ┃ main 1 this 0 -1
1100-1150 ms ┃ main 0 this 1 +1
1950-2000 ms ┃ main 1 this 0 -1

1020 steps (queue-hop)
Cumulative STSO time: main 8138ms → this run 9272ms (Δ +1134ms, +14%)
2000-2500 ms ███████████████████████┃ main 1 this 1 +0
2500-3000 ms ┃███████████████████████ main 1 this 0 -1
3000-3500 ms ███████████████████████┃ main 1 this 1 +0
3500-4000 ms ░░░░░░░░░░░░░░░░░░░░░░░┃ main 0 this 1 +1

📜 Previous results (1)
960fef2
Thu, 30 Jul 2026 23:59:06 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1284 (+27%) 🔻1386 🔴 (+23%) 🔻1408 🔴 (+21%) 🔻1470 🔴 (+16%) 🔻30
TTFSstream341 (+66%) 🔻1425 🔴 (+29%) 🔻1462 🔴 (+29%) 🔻1677 🔴 (+3.4%)30
TTFShook + stream1497 (+13%)1695 🔴 (+18%) 🔻1770 🔴 (+21%) 🔻1912 🔴 (+17%) 🔻30
STSO1020 steps (inline)171 (-7.1%)483 (-6.9%)563 (-2.1%)826 (+2.2%)1016
STSO1020 steps (queue-hop)1772 (-12%)3369 (-28%) 💚3369 (-28%) 💚3369 (-28%) 💚3
WO1020 steps420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)420728 (-6.7%)1
SLstream latency85 (-24%) 💚181 🔴 (+9.0%)198 🔴 (-5.3%)563 🔴 (+45%) 🔻30
SOstream overhead (text)122 (-7.6%)170 (-41%) 💚194 (-61%) 💚282 (-90%) 💚30
SOstream overhead (structured)110 (-17%) 💚168 (-40%) 💚194 (-50%) 💚218 (-86%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Jul 30, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
▲ Vercel Production (1 failed)
sveltekit (1 failed):
  • writableForwardedFromWorkflowWorkflow | wrun_41KYWGR9SF0GHY8Y185EHHPXSX | 🔍 observability
E2E Test Summary
Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production145412391694
✅ 💻 Local Development162102271848
✅ 📦 Local Production162102271848
✅ 🐘 Local Postgres162102271848
✅ 🪟 Windows15400154
✅ 📋 Other102002121232
✅ vercel-multi-region270027
Total7518111328651
Details by Category
❌ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126028
✅ example126028
✅ express126028
✅ fastify126028
✅ hono126028
✅ nextjs-turbopack15103
✅ nextjs-webpack15103
✅ nitro126028
✅ nuxt126028
❌ sveltekit14419
✅ vite126028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128026
✅ express-stable128026
✅ fastify-stable128026
✅ hono-stable128026
✅ nextjs-turbopack-canary135019
✅ nextjs-turbopack-stable15400
✅ nextjs-webpack-canary135019
✅ nextjs-webpack-stable15400
✅ nitro-stable128026
✅ nuxt-stable128026
✅ sveltekit-stable14707
✅ vite-stable128026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15400
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128026
✅ e2e-local-dev-tanstack-start-128026
✅ e2e-local-postgres-nest-stable128026
✅ e2e-local-postgres-tanstack-start-128026
✅ e2e-local-prod-nest-stable128026
✅ e2e-local-prod-tanstack-start-128026
✅ e2e-vercel-prod-nest126028
✅ e2e-vercel-prod-tanstack-start126028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
CopilotAI
reviewed
Jul 30, 2026
View reviewed changes
CopilotAI
left a comment


Copy link


Copy Markdown

Contributor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview
Prevents changeset-release/main (changesets “Version Packages” PR branch) from triggering Vercel preview deployments that can overwrite per-SHA commit statuses and break production e2e runs, and updates CI to run those PRs’ Vercel e2e lanes against main’s production deployment instead.
Changes:
  • Disable Vercel deployments for changeset-release/main across all Vercel project roots via git.deploymentEnabled.
  • Update Vercel e2e CI to resolve the base SHA’s production deployment for changeset-release/* PRs and run downstream as production.
  • Skip deployment-dependent smoke/benchmark jobs for changeset-release/* PRs and document the invariant in AGENTS.md.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
FileDescription
workbench/vite/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/tanstack-start/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/swc-playground/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/sveltekit/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nuxt/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v3/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nitro-v2/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-webpack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nextjs-turbopack/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/nest/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/hono/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/fastify/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/express/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/example/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
workbench/astro/vercel.jsonDisable changeset-release/main deployments for this Vercel project root.
tarballs/vercel.jsonDisable changeset-release/main deployments for the tarballs project.
packages/web/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for packages/web.
docs/vercel.jsonAdd Vercel config to disable changeset-release/main deployments for docs.
AGENTS.mdDocument the “never deploy changeset-release/main” invariant and CI expectations.
.github/workflows/tests.ymlSpecial-case changeset-release/* PRs to resolve base SHA production deploy and run e2e as production.
.github/workflows/tarballs-checks.ymlSkip preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/docs-checks.ymlSkip docs preview-smoke on changeset-release/* PRs (no preview deploy exists).
.github/workflows/benchmarks.ymlSkip benchmark PR jobs on changeset-release/* PRs (no preview deploy; results meaningless).
.github/scripts/resolve-production-deployment.mjsAdd helper script to poll Vercel API for the base SHA’s READY production deployment and emit wait-action-compatible outputs.
.changeset/skip-changeset-release-deploys.mdAdd changeset entry documenting this CI/deployment behavior change.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated


Comment on lines

+58
 to 
+60

const timeoutMs = Number(timeoutSeconds) * 1000;
const pollIntervalMs = Number(pollIntervalSeconds) * 1000;
const deadline = Date.now() + timeoutMs;
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script was removed entirely in 5daef6a — the lane now reuses vercel/wait-for-deployment-action (tokenless, sha pinned to the base SHA) per the maintainer suggestion above, so there's no TIMEOUT_SECONDS/POLL_INTERVAL_SECONDS parsing left to validate. Good catch regardless — the NaN-deadline hazard was real in the deleted code.
TooTallNate
TooTallNate
approved these changes
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed alongside vercel/wait-for-deployment-action#6 against the incident analysis from the Slack thread. Part 1 removes the same-SHA preview deployments that clobber the per-SHA commit status (the actual trigger), and Part 2's wiring is consistent: I traced every github.head_ref condition (empty on push/workflow_dispatch, so main and dispatch paths are untouched), the waitForDeployment || prodDeployment output fallbacks, and the WORKFLOW_VERCEL_ENV/VERCEL_WORKFLOW_SERVER_URL expressions — all correct for the three cases (main push, normal PR, changeset PR). All six workflows that wait on deployments are accounted for (tests, benchmarks, docs-checks, tarballs-checks handled here; event-log-race-repro is label-gated; dispatch-front-workflow-release-pr is disabled). Confirmed packages/web/vercel.json is excluded from @workflow/web's files allowlist.
On the red checks: the Biome failure is inherited from main (organizeImports in packages/core/src/runtime.ts and packages/core/src/runtime/step-executor.ts — files this PR doesn't touch; main's Lint run fails identically), and the E2E Vercel Prod (example) failure is a single flaky hook-disposal timeout (Timed out waiting for hook ... to be disposed) on the unchanged normal-PR path. Neither is caused by this PR.
No blocking issues. One latent edge flagged inline in the resolve script around skipped/canceled production builds; fine as a follow-up since no project in this repo currently uses an ignored build step.
Comment thread.github/scripts/resolve-production-deployment.mjs
Outdated

const PAGE_SIZE = 100;
// States a deployment can never leave. Reaching one of these for the expected
// SHA means the production build failed; there is nothing to wait for.
const TERMINAL_FAILURE_STATES = new Set(['ERROR', 'CANCELED', 'DELETED']);
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (latent edge): when Vercel skips a production build ("Skipped – Not affected" via an ignored build step), the deployment record for that SHA ends up CANCELED — which this treats as a terminal failure and exits 1 immediately. The wait action handles that same case by treating the GitHub inactive status as ready and surfacing the still-live previous URL; this script has no equivalent, so a changeset-release lane would go red even though production is healthy.
No project in this repo has an ignoreCommand today, so this can't fire yet — but if one ever gains an ignored build step, every Version Packages PR whose base commit didn't affect that project will fail here. Worth a follow-up (e.g. on all-CANCELED matches, fall back to the most recent READY production deployment — safe in exactly this case because "skipped" means the code at the base SHA is identical to what's already deployed), or at least a comment in AGENTS.md's new section noting ignored build steps are incompatible with this resolution strategy.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by adopting your tests.yml suggestion (5daef6a): the script is gone, and the lane now goes through the wait action, which treats the inactive GitHub Deployment status from "Skipped – Not affected" as ready and surfaces the still-live URL. So a project gaining an ignored build step no longer reds out the Version Packages lane. The AGENTS.md note about this branch's special treatment stays, minus the script reference.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Open

TooTallNate
TooTallNate
reviewed
Jul 31, 2026
View reviewed changes
@TooTallNateTooTallNate
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to my approval, after a policy note from the team: we're moving away from using Vercel access tokens as GitHub secrets, and wait-for-deployment-action's founding design goal was to not need one. That doesn't change my verdict on this PR — Part 1 (disabling changeset-release/main deployments) is the real fix and is token-free, and the new script's VERCEL_TOKEN usage doesn't add a new secret since VERCEL_LABS_TOKEN is already in these same job steps as WORKFLOW_VERCEL_AUTH_TOKEN. But there's a simpler, token-free alternative for Part 2 worth considering (inline) that would also delete the 192-line script and resolve my earlier comment about skipped builds.
Comment thread.github/workflows/tests.yml
Outdated

 id: prodDeployment
 if: ${{ startsWith(github.head_ref, 'changeset-release/') }}
 env:
 VERCEL_TOKEN: ${{ secrets.VERCEL_LABS_TOKEN }}
@TooTallNateTooTallNateJul 31, 2026

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (non-blocking, but aligns with the no-Vercel-token-secrets direction): once Part 1 of this PR lands, the base SHA (main's HEAD) can no longer have a same-SHA changeset-release/main preview deployment — which is precisely the only thing that made the per-SHA commit status ambiguous. That means the changeset lane could reuse the existing tokenless wait action instead of this script + VERCEL_TOKEN:
- name: Waiting for the Vercel deployment (changeset-release PR)id: prodDeploymentif: ${{ startsWith(github.head_ref, 'changeset-release/') }}uses: vercel/wait-for-deployment-action@0e2b0c5c5cce31f1648108aeec56467187aca037with:
project-slug: ${{ matrix.app.project-slug }}timeout: 1000check-interval: 15environment: productionsha: ${{ github.event.pull_request.base.sha }}
The action already supports an explicit sha input, polls GitHub's env-scoped Deployment for Production – <slug>, and resolves the ID from the commit status — which is safe for main HEAD SHAs post-Part-1 (a same-SHA PR-branch preview of an unrelated PR can't exist for a commit that's already on main). It would also inherit the action's inactive/skipped-build handling, addressing my other comment about CANCELED deployments, and drop the new script entirely.
Caveat: the theoretical residual ambiguity is another project/branch deploying the same SHA, which Part 1 rules out for the only branch that ever does this. If you prefer the exactness of the Vercel API lookup, keeping the script is defensible since the secret already exists in this job — but flagging the tokenless option given the policy direction.
@pranaygppranaygpJul 31, 2026

Copy link


Copy Markdown

ContributorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adopted in 5daef6a, with one verification first: confirmed the pinned ref (0e2b0c5c) does ship the sha input by reading its action.yml at that commit. Both lanes (e2e matrix + multi-region) now run the wait action a second way — environment: production, sha: ${{ github.event.pull_request.base.sha }} — and the 192-line script plus its VERCEL_TOKEN usage are deleted. The step comment and AGENTS.md now record why the commit status is trustworthy for exactly this lane: with changeset-release/main no longer deployed (Part 1 of this PR), main SHAs can't be deployed to a second environment of these projects, and that branch was the only one that ever deployed a commit main also deployed. WORKFLOW_VERCEL_ENV=production and the empty VERCEL_WORKFLOW_SERVER_URL are unchanged — those are about attribution, not resolution.
@TooTallNateTooTallNate
mentioned this pull request
Jul 31, 2026
 Merged

@pranaygp
ci: resolve changeset-release e2e deployments with the wait action, t…
5daef6a
…okenless
Per review: with changeset-release/main no longer deployed, main SHAs
can never again be deployed to a second environment of these projects,
so the per-SHA commit status the action reads is unambiguous for
exactly this lane. Reuse vercel/wait-for-deployment-action with
environment: production and sha pinned to the PR base SHA instead of
the Vercel-API polling script, drop the script and its VERCEL_TOKEN
usage, and inherit the action's inactive/skipped-build handling.
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@pranaygp
pranaygp
merged commit 11dc036
into
mainJul 31, 2026

105 of 107 checks passed

@pranaygp
pranaygp
deleted the

pgp/skip-changeset-release-deploys

branch
July 31, 2026 17:09
@github-actions
github-actionsBot
commented
Jul 31, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for 11dc036 (AI decision).
This is CI plumbing that fixes a problem specific to main: a same-SHA preview deployment of changeset-release/main clobbering the production commit status that vercel/wait-for-deployment-action reads. On stable, deployment waits already resolve to preview (only refs/heads/main maps to production in stable's tests.yml), and the new vercel.json keys name changeset-release/main, not changeset-release/stable, so the fix would deliver no benefit there. Worse, the new startsWith(github.head_ref, 'changeset-release/') gates would match changeset-release/stable PRs and send them waiting for a production deployment of a stable base SHA that never exists — hanging or skipping CI on stable's own version PRs — so backporting would be a regression rather than a stability fix.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
11dc036854749d154e73024d0109c3bfce462308

pranaygp added a commit
that referenced
this pull request
Jul 31, 2026
@pranaygp
Merge remote-tracking branch 'origin/main' into pgp/runinput-environm…
32d6e27
…ent-guard
* origin/main:
ci: stop deploying changeset-release/main, run its e2e against production (#3243)
fix(world-local): bound stalled queue deliveries (#3255)
Sort imports in runtime.ts and step-executor.ts (#3241)
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@TooTallNateTooTallNateTooTallNate approved these changes
Copilot code reviewCopilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@pranaygp@TooTallNate