Uh oh!
There was an error while loading. Please reload this page.
[core] De-flake dev.test.ts cleanup hook timeout - #2416
Merged
Conversation
The `should rebuild on adding workflow file` test tears down by restoring an edited api file and deleting an added workflow file — both trigger dev server rebuilds — then runs two *sequential* prewarm() calls. Each prewarm fetch is bounded at 5s via AbortSignal.timeout, so the cleanup's worst case is ~2×5s plus file IO, right at vitest's 10s default hook timeout. When the dev server is mid-rebuild the fetches stall toward their bound and the hook trips "Hook timed out in 10000ms" — flaky by construction, mainly on the webpack/eager (stable lazyDiscovery-disabled) and Windows lanes. Give the beforeAll/afterEach hooks an explicit budget of 4× the prewarm fetch bound (20s), derived from a shared constant rather than a magic number. The fetches remain hard-bounded so cleanup still can't hang indefinitely; the hook just gets headroom over its bounded worst case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: a1e36b1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
workflow with 1 step💻 Local Development
▲ Production (Vercel)
workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
Stream Benchmarks(includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
❌ Some benchmark jobs failed:
Check the workflow run for details. |
VaguelySerious
marked this pull request as ready for review
June 14, 2026 11:30
Uh oh!
There was an error while loading. Please reload this page.
github-actionsBot
added a commit
that referenced
this pull request
Jun 15, 2026
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
Contributor
Backport PR opened against |
VaguelySerious pushed a commit
that referenced
this pull request
Jun 15, 2026
pranaygp added a commit
that referenced
this pull request
Jun 16, 2026
* origin/main: Small detail panel cleanup (#2459) Fix lazy Next workflow HMR (#2438) Prevent peer dependency-only major bumps (#2437) fix(changesets): only major-bump peer dependents when out of range (#2439) Version Packages (beta) (#2428) otel: explicit traceparent injection + linked-trace mode for bounded per-invocation traces (#2363) [next] Clarify `serverExternalPackages` warning (#2417) Add .swc gitignore handling to builder (#2427) Version Packages (beta) (#2390) [ci] Increase dev.test.ts cleanup hook timeout (#2416) [world-vercel] Switch event endpoints to v4 wire format (#2055) docs: document run idempotency (#2011) Render attr_set events and run attributes in observability UI (#2393) [ci] Fix backport job model slug (#2403) [ci] Comment on PR when backport fails, revert to use opus 4.8 (#2400) Update queue client to 0.3.1 (#2399) fix(deps): upgrade esbuild to 0.28.1 (GHSA-gv7w-rqvm-qjhr) (#2395) test: e2e coverage for run-idempotency conflict-handling strategies (#2387) # Conflicts: # pnpm-lock.yaml
pranaygp added a commit
that referenced
this pull request
Jun 16, 2026
* origin/main: fix(core): make deploymentId 'latest' a no-op in non-Vercel worlds (#2397) fix(deps): upgrade astro to 6.4.6 to resolve CVE-2026-54299 (#2457) Small detail panel cleanup (#2459) Fix lazy Next workflow HMR (#2438) Prevent peer dependency-only major bumps (#2437) fix(changesets): only major-bump peer dependents when out of range (#2439) Version Packages (beta) (#2428) otel: explicit traceparent injection + linked-trace mode for bounded per-invocation traces (#2363) [next] Clarify `serverExternalPackages` warning (#2417) Add .swc gitignore handling to builder (#2427) Version Packages (beta) (#2390) [ci] Increase dev.test.ts cleanup hook timeout (#2416) [world-vercel] Switch event endpoints to v4 wire format (#2055)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
E2E Local Dev Testslanes intermittently fail with:Most often on the webpack / eager-discovery (stable
lazyDiscovery disabled) and Windows lanes. (example run)Root cause
should rebuild on adding workflow fileadds a workflow file and edits the api-file import. ItsafterEachtherefore takes the expensive cleanup path: it restores the api file and deletes the added workflow file — both trigger dev-server rebuilds — then runs two sequentialprewarm()calls.Each prewarm is
Promise.all([fetch('/'), fetch('/api/chat')]), each fetch hard-bounded at 5s viaAbortSignal.timeout. So the cleanup's worst case is ~2×5s of fetches plus file IO — right at vitest's default 10s hook timeout. The teardown runs precisely when the dev server is mid-rebuild (it's reverting the file add + import), so the prewarm fetches stall toward their 5s bound and the hook blows its 10s budget. The bounded fetches were added so cleanup "cannot hang," but the hook timeout was never raised above the sum of those bounds → marginal-by-construction → flaky.This is independent of any product code — it reproduces on
main/stable(the lane runs the repo'sdev.test.tsagainst the stable SDK).Fix
Give the
beforeAll/afterEachhooks an explicit budget of 4× the prewarm fetch bound (20s), derived from a single shared constant (no duplicated magic number, per repo test conventions):The fetches remain hard-bounded by
AbortSignal.timeout, so the cleanup still can't hang indefinitely (worst case ≤ ~11s) — the hook just gets comfortable headroom over its bounded worst case. The two existing 5s fetch literals now reference the constant.Validation
A timing flake on a rebuild-bound dev server isn't deterministically reproducible locally, so this is validated by the bounded-work arithmetic: worst case ≈ 2×5s + file IO ≈ ≤11s, now under a 20s budget instead of the 10s default. CI exercises it on the affected lanes.