Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); Avoid resolving run data for background steps by NathanColosimo · Pull Request #2993 · vercel/workflow · GitHub
Skip to content

Avoid resolving run data for background steps - #2993

Merged
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run
Jul 20, 2026
Merged

Avoid resolving run data for background steps#2993
NathanColosimo merged 2 commits into
mainfrom
codex/background-step-unresolved-run

Conversation

@NathanColosimo

@NathanColosimoNathanColosimo commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load only run metadata before executing a background step
  • keep the existing terminal-state guard and replay behavior unchanged

Why

Background-step setup only consumes run status, timing, deployment, attributes, and spec version. Resolving input and output data at this point is unnecessary, especially for fan-out workloads and runs with large inputs.

Impact

Reduces data loading and response payload size during background-step startup without changing execution semantics.

Validation

  • pnpm turbo run build --filter=@workflow/core...
  • pnpm test in packages/core — 1,483 passed, 3 expected failures
  • pnpm typecheck
  • targeted Biome check for the changed files

@changeset-bot

changeset-botBot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3061de4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/corePatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
✅ vercel-multi-region270027
Total7378010648442

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit 3061de4 · Mon, 20 Jul 2026 18:07:36 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1257 (+16%)12041278 🔴1299 🔴1330 🔴30
TTFSstream1268 (+23%)12271261 🔴1275 🔴1566 🔴30
TTFShook + stream1535 (+19%)14841571 🔴1602 🔴2085 🔴30
STSO1020 steps (1-20)236 (+2.1%)179264 🔴314 🔴340 🔴19
STSO1020 steps (101-120)247 (-2.4%)196251 🔴300 🔴660 🔴19
STSO1020 steps (1001-1020)555 (-1.3%)490546 🔴721 🔴965 🔴19
WO1020 steps389373 (-1.6%)3893733893733893733893731
SLstream latency139 (+17%)102143 🔴170 🔴328 🔴30
📜 Previous results (1)

f8f1236

Fri, 17 Jul 2026 23:45:15 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P10 (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1027 (-2.4%)2361299 🔴1423 🔴1521 🔴30
TTFSstream1019 (+1.8%)2521286 🔴1321 🔴1423 🔴30
TTFShook + stream1374 (+8.5%)5631557 🔴1607 🔴1681 🔴30
STSO1020 steps (1-20)302 (+27%)199351 🔴463 🔴755 🔴19
STSO1020 steps (101-120)280 (+8.5%)216290 🔴369 🔴514 🔴19
STSO1020 steps (1001-1020)557 (+11%)496548 🔴736 🔴777 🔴19
WO1020 steps419944 (+6.5%)4199444199444199444199441
SLstream latency164 (+8.3%)118182 🔴233 🔴359 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt)

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the P10 column shows the warm-start floor for comparison.

@NathanColosimo
NathanColosimoforce-pushed the codex/background-step-unresolved-run branch from bcf3d68 to f8f1236CompareJuly 17, 2026 23:28
@NathanColosimo
NathanColosimo marked this pull request as ready for review July 17, 2026 23:29
@NathanColosimo
NathanColosimo requested review from a team and ijjk as code ownersJuly 17, 2026 23:29
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/core/src/runtime.ts
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo enabled auto-merge (squash) July 20, 2026 18:43
@NathanColosimo
NathanColosimo merged commit 6d1d700 into mainJul 20, 2026
171 of 173 checks passed
@NathanColosimo
NathanColosimo deleted the codex/background-step-unresolved-run branch July 20, 2026 18:51
@github-actionsgithub-actionsBot mentioned this pull request Jul 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No backport to stable for 6d1d700 (AI decision).

The change optimizes the background-step startup path inside workflowEntrypoint in packages/core/src/runtime.ts, but that path (the incomingStepId/incomingStepName background-step handling, bgRun guard, and inline replay after a background step) does not exist on stable — verified via git show origin/stable:packages/core/src/runtime.ts, which has no background-step logic at all. Since the commit builds entirely on main-only behavior, there is nothing to backport.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

6d1d7006cf442c715c464ec2b8c80a21d1c90b01

pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main: (21 commits)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
perf(core): cache port discovery in step invocations for self-hosted worlds (#2996)
feat(web-shared): Alt+hover span measurement in the new trace viewer (#2985)
fix(world-postgres): throw EntityConflictError on duplicate run_created (#2983)
[ci] Run benchmarks in-deployment to avoid proxy overhead (#2967)
Enable additional perf optimizations when correctness guarantees are met (#2970)
perf(core): prepare replay payloads concurrently (#2980)
Fix dotted tsconfig alias workflow discovery (#2963)
Adjust helper position on trace viewer (#2968)
...
pranaygp added a commit that referenced this pull request Jul 21, 2026
* origin/main:
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
fix(core): batch stream writes via writeMulti (#2995)
pranaygp added a commit that referenced this pull request Jul 22, 2026
* origin/main: (162 commits)
Implement `max_events` per run limit (#2986)
[core] Enforce maxRetries for steps that time out (#3035)
[world-vercel] Idempotent retry policy for stream close (5xx retriable) (#3038)
[world] Guard hook_received against a concurrent run termination (#2987)
docs: fix stale/incorrect v5 API reference details (#3017)
Default WORKFLOW_PRECONDITION_GUARD on (#2946)
docs: replace migration guides with a Comparisons section (#2676)
feat(core): add experimental Hook minimum retention (#2865)
test: regression coverage for hook.resume() from isolated route bundles (o2flow beta.26 incident) (#3001)
docs(agents): note lint/format/typecheck are advisory, not blocking (#2886)
Retry transient connection timeouts (#3013)
fix(world-vercel): append caller User-Agent products instead of discarding them (#2998)
[ci] Enable NestJS e2e-vercel-prod and add to docs as "experimental" (#3011)
[ci] Benchmark comment: Best column + best/p75/p99 deltas (drop Avg/P10) (#3005)
docs: fall back to first child page for sidebar folders without an index (#3009)
[nest] Fix NestJS Vercel build output (#2988)
Avoid resolving run data for background steps (#2993)
chore(docs): update @vercel/geistdocs to 1.14.0 (#3002)
fix(docs): add version-switcher fallback redirects for pages missing in one version (#3003)
ci: update opencode to 1.18.4 and switch backport AI model to claude-fable-5 (#3006)
...
# Conflicts:
#	docs/content/docs/v4/deploying/meta.json
#	docs/content/docs/v5/deploying/meta.json
#	packages/core/src/runtime.ts
#	packages/world-local/src/index.ts
#	packages/world-postgres/src/index.ts
#	packages/world/src/events.ts
#	packages/world/src/interfaces.ts
#	packages/world/src/recovery.ts
#	workbench/nest/src/main.ts
#	workbench/sveltekit/src/hooks.server.ts
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@NathanColosimo@VaguelySerious@karthikscale3