Deprecate world.runs.list for observability - #3404

Merged
karthikscale3 merged 3 commits into
mainfrom
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged

Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom
kk/deprecate-runs-list-observability

Conversation

@karthikscale3

@karthikscale3karthikscale3 commented Aug 9, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.

Changes

  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.

Validation

  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI

@karthikscale3
karthikscale3 requested a review from a team as a code ownerAugust 9, 2026 19:31
@changeset-bot

changeset-botBot commented Aug 9, 2026•
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: fb10fea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 9, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm

@github-actions

github-actionsBot commented Aug 9, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)

 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1
📜 Previous results (1)

e760414

Sun, 09 Aug 2026 19:48:17 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology

The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000

All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.

Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.

@github-actions

github-actionsBot commented Aug 9, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

❌ Some tests failed

❌ Failed E2E Tests

📋 Other (2 failed)

e2e-local-prod-tanstack-start-node (1 failed):

  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B

e2e-vercel-prod-nest-quickjs (1 failed):

  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE

E2E Test Summary

Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category

✅ ▲ Vercel Production

AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028

✅ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600

❌ 📋 Other

AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@karthikscale3
karthikscale3 merged commit f1ef0cb into mainAug 12, 2026
162 of 166 checks passed
@karthikscale3
karthikscale3 deleted the kk/deprecate-runs-list-observability branch August 12, 2026 21:36
@github-actions

github-actionsBot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

No backport to stable for f1ef0cb (AI decision).

This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.

To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:

f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit that referenced this pull request Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp mentioned this pull request Aug 13, 2026
Merged
VaguelySerious added a commit that referenced this pull request Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
Deprecate world.runs.list for observability - #3404
Merged
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Aug 12, 2026
Merged
Deprecate world.runs.list for observability#3404
karthikscale3 merged 3 commits into
mainfrom 
kk/deprecate-runs-list-observability
Conversation
@karthikscale3
@karthikscale3karthikscale3
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Documents that observability and inspection callers should use world.analytics.runs.list() instead of the canonical storage list. Operational and payload-bearing callers remain supported.
Changes
  • Clarifies the distinction in the @workflow/world interface and Vercel world implementation JSDoc.
  • Adds v5 Storage guidance and a warning at world.runs.list().
  • Adds a reciprocal link from the v5 Analytics reference.
  • Uses an empty changeset because this is documentation-only and should not publish package patch releases.
Validation
  • git diff --check
  • pnpm changeset status accepts the empty changeset
  • Full docs build and link validation run in CI
@karthikscale3
karthikscale3
requested a review
from a team
as a code ownerAugust 9, 2026 19:31
@changeset-bot
changeset-botBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: fb10fea
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 0 packages
When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 9, 2026 8:10pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 9, 2026 8:10pm
example-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-astro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-express-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-fastify-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-hono-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nestjs-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nitro-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-nuxt-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-python-workflowErrorErrorAug 9, 2026 8:10pm
workbench-sveltekit-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workbench-vite-workflowReadyReadyPreviewAug 9, 2026 8:10pm
workflow-docsReadyReadyPreview, v0Aug 9, 2026 8:10pm
workflow-swc-playgroundReadyReadyPreviewAug 9, 2026 8:10pm
workflow-tarballsReadyReadyPreviewAug 9, 2026 8:10pm
workflow-webReadyReadyPreviewAug 9, 2026 8:10pm
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
📊 Workflow Benchmarks
commit fb10fea · Sun, 09 Aug 2026 20:25:09 GMT · run logs
Backend: vercel · app: nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep216 (-76%) 💚1350 🔴 (+19%) 🔻1370 🔴 (+15%) 🔻1472 🔴 (-4.4%)30
TTFSstream263 (+12%)1462 🔴 (+33%) 🔻1497 🔴 (+33%) 🔻1756 🔴 (+52%) 🔻30
TTFShook + stream392 (-15%) 💚1689 🔴 (+14%)1742 🔴 (+4.0%)1838 🔴 (+1.4%)30
STSO1020 steps (inline)96 (+4.3%)153 (+4.8%)178 (+4.1%)412 (+47%) 🔻1019
WO1020 steps152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)152723 (+7.0%)1
SLstream latency101 (+6.3%)166 🔴 (+15%) 🔻247 🔴 (+43%) 🔻903 🔴 (+145%) 🔻30
SOstream overhead (text)122 (±0%)237 (+19%) 🔻375 (+65%) 🔻707 (+67%) 🔻30
SOstream overhead (structured)129 (+4.0%)276 🔴 (+36%) 🔻390 (+76%) 🔻821 (+125%) 🔻30
📈 STSO distribution vs main (inline / queue-hop histograms)
1020 steps (inline)
Cumulative STSO time: main 141722ms → this run 151379ms (Δ +9657ms, +7%)
 50-100 ms ┃ main 3 this 1 -2
100-150 ms █████████████████████┃██ main 794 this 721 -73
150-200 ms ██████┃ main 183 this 241 +58
200-250 ms ┃ main 24 this 25 +1
250-300 ms ┃ main 6 this 12 +6
300-350 ms ┃ main 3 this 3 +0
350-400 ms ┃ main 3 this 5 +2
400-450 ms ┃ main 1 this 4 +3
450-500 ms ┃ main 1 this 1 +0
500-550 ms ┃ main 1 this 2 +1
550-600 ms ┃ main 0 this 1 +1
600-650 ms ┃ main 0 this 1 +1
700-750 ms ┃ main 0 this 1 +1
800-850 ms ┃ main 0 this 1 +1

📜 Previous results (1)
e760414
Sun, 09 Aug 2026 19:48:17 GMT · run logs
vercel / nextjs-turbopack
MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1274 (+40%) 🔻1355 🔴 (+20%) 🔻1368 🔴 (+15%)1437 🔴 (-6.7%)30
TTFSstream1262 (+437%) 🔻1328 🔴 (+21%) 🔻1346 🔴 (+20%) 🔻1426 🔴 (+24%) 🔻30
TTFShook + stream494 (+6.9%)1625 🔴 (+9.8%)1733 🔴 (+3.5%)1780 🔴 (-1.8%)30
STSO1020 steps (inline)90 (-2.2%)126 (-14%)144 (-16%) 💚298 (+6.0%)1019
WO1020 steps127851 (-10%)127851 (-10%)127851 (-10%)127851 (-10%)1
SLstream latency88 (-7.4%)113 🔴 (-22%) 💚120 🔴 (-31%) 💚139 🔴 (-62%) 💚30
SOstream overhead (text)100 (-18%) 💚168 (-16%) 💚205 (-9.7%)451 (+6.6%)30
SOstream overhead (structured)102 (-18%) 💚176 (-13%)215 (-3.2%)232 (-36%) 💚30
ℹ️ Metric definitions & methodology
The collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: █ is main, ┃ marks where this run lands, ░ bridges the gap when this run has more samples in a bucket.
Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.
Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window)
Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost
🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000
All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor (clientStart) right before start(), so the CI runner’s request and its path through api.vercel.com sit outside every measured window. TTFS = in-deployment start() → first step body (turbo uses the in-process fast path, non-turbo the dispatch path), and includes the VQS dispatch hop plus any /flow cold start. STSO/WO are measured between step bodies on the deployment. SL is measured inside the workflow (parallel reader/writer steps), so it no longer includes the api.vercel.com read path.
Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the /flow invocation for a large fraction of runs, inflating P75+; the Best column shows the fastest (warm-start) sample for comparison.
@github-actions
github-actionsBot
commented
Aug 9, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
❌ Some tests failed
❌ Failed E2E Tests
📋 Other (2 failed)
e2e-local-prod-tanstack-start-node (1 failed):
  • webhookWorkflow | wrun_41KZM2CTS00GZMGY9B9VFRYT3B
e2e-vercel-prod-nest-quickjs (1 failed):
  • startFromWorkflow - calling start() directly inside a workflow function with hook communication | wrun_41KZM30DGF0GX7HWTRVSC79WEE
E2E Test Summary
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production295404783432
✅ 💻 Local Development329004543744
✅ 📦 Local Production329004543744
✅ 🐘 Local Postgres329004543744
✅ 🪟 Windows31200312
❌ 📋 Other207024242496
✅ vercel-multi-region270027
Total152332226417499
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro-node128028
✅ astro-quickjs128028
✅ example-node128028
✅ example-quickjs128028
✅ express-node128028
✅ express-quickjs128028
✅ fastify-node128028
✅ fastify-quickjs128028
✅ hono-node128028
✅ hono-quickjs128028
✅ nextjs-turbopack-node15303
✅ nextjs-turbopack-quickjs15303
✅ nextjs-webpack-node15303
✅ nextjs-webpack-quickjs15303
✅ nitro-node128028
✅ nitro-quickjs128028
✅ nuxt-node128028
✅ nuxt-quickjs128028
✅ sveltekit-node14709
✅ sveltekit-quickjs14709
✅ vite-node128028
✅ vite-quickjs128028
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable-node130026
✅ astro-stable-quickjs130026
✅ express-stable-node130026
✅ express-stable-quickjs130026
✅ fastify-stable-node130026
✅ fastify-stable-quickjs130026
✅ hono-stable-node130026
✅ hono-stable-quickjs130026
✅ nextjs-turbopack-canary-node137019
✅ nextjs-turbopack-canary-quickjs137019
✅ nextjs-turbopack-stable-node15600
✅ nextjs-turbopack-stable-quickjs15600
✅ nextjs-webpack-canary-node137019
✅ nextjs-webpack-canary-quickjs137019
✅ nextjs-webpack-stable-node15600
✅ nextjs-webpack-stable-quickjs15600
✅ nitro-stable-node130026
✅ nitro-stable-quickjs130026
✅ nuxt-stable-node130026
✅ nuxt-stable-quickjs130026
✅ sveltekit-stable-node14907
✅ sveltekit-stable-quickjs14907
✅ vite-stable-node130026
✅ vite-stable-quickjs130026
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack-node15600
✅ nextjs-turbopack-quickjs15600
❌ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable-node130026
✅ e2e-local-dev-nest-stable-quickjs130026
✅ e2e-local-dev-tanstack-start-node130026
✅ e2e-local-dev-tanstack-start-quickjs130026
✅ e2e-local-postgres-nest-stable-node130026
✅ e2e-local-postgres-nest-stable-quickjs130026
✅ e2e-local-postgres-tanstack-start-node130026
✅ e2e-local-postgres-tanstack-start-quickjs130026
✅ e2e-local-prod-nest-stable-node130026
✅ e2e-local-prod-nest-stable-quickjs130026
❌ e2e-local-prod-tanstack-start-node129126
✅ e2e-local-prod-tanstack-start-quickjs130026
✅ e2e-vercel-prod-nest-node128028
❌ e2e-vercel-prod-nest-quickjs127128
✅ e2e-vercel-prod-tanstack-start-node128028
✅ e2e-vercel-prod-tanstack-start-quickjs128028
✅ vercel-multi-region
AppPassedFailedSkipped
✅ nextjs-turbopack2700
📋 View full workflow run
@karthikscale3
karthikscale3
merged commit f1ef0cb
into
mainAug 12, 2026

162 of 166 checks passed

@karthikscale3
karthikscale3
deleted the

kk/deprecate-runs-list-observability

branch
August 12, 2026 21:36
@github-actions
github-actionsBot
commented
Aug 12, 2026

Copy link


Copy Markdown

Contributor
No backport to stable for f1ef0cb (AI decision).
This commit documents a deprecation that steers callers toward world.analytics.runs.list(), an API that does not exist on stable — git grep analytics origin/stable -- packages/world/src/interfaces.ts returns nothing and docs/content/docs/v5/api-reference/workflow-runtime/world/analytics.mdx is absent there. It is not a correction of anything wrong on stable; backporting it would produce JSDoc and docs pointing at a non-existent replacement and a broken cross-link.
To override, re-run the Backport to stable workflow manually via workflow_dispatch and paste this commit SHA into the ref input:
f1ef0cbf037631fa1a0978d0acb0533ef5d704dd

shalabhc added a commit
that referenced
this pull request
Aug 13, 2026
@vercel@shalabhc
Deprecate world.runs.list for observability (#3404)
0916dc7
* Deprecate storage run listing for observability
* Document analytics run listing guidance
Co-Authored-By: shalabhchaturvedi-7802 <shalabh.chaturvedi@vercel.com>
@pranaygppranaygp
mentioned this pull request
Aug 13, 2026
 Merged

VaguelySerious added a commit
that referenced
this pull request
Aug 13, 2026
@VaguelySerious@claude
Say what actually makes a stale replay safe, and restore a doc comment
88553f8
Review feedback. The safety argument said the fence does it — "every durable
write the stale replay attempts is rejected with 412" — and no shipped World
does that for any run this build can replay. Someone verifying the claim
against world-postgres would find no fence and conclude the gate is unsound.
What does it is three properties together, none of which is a rejection. A
reader's log is a prefix of the run's log rather than a prefix with a hole in
it, because positions are allocated at the commit. Replay is deterministic on a
prefix, so a shorter log means a run that has not caught up, never one that
decides differently. And every write reports what it missed: it names the
position it was derived from and the World returns the events it was pushed
past. That is why the capability could be deleted rather than universalized —
the need for it was removed, not spread around.
412 keeps its error type, its docs and the runtime's handling of it, for a World
that allocates positions away from the commit and would rather refuse than
report. Nothing in this repository throws it today, and the docs now say so.
Also from review:
- Restores the `runs.list` deprecation comment from #3404, reverted here by a
rebase, and updates the doc comments that still reasoned in guarded/unguarded
terms or promised ULID runs stay replayable.
- `suppressOptimisticStart` now states the reason that fires everywhere: the
writer that loses a claim has already run a body it does not own, and
awaiting the claim is what stops that.
- The sim world no longer mints a boundary position for `events.create(null,
…)`. The Storage contract allows it, and minting under the null key would
serve slots from a shared bucket while the generated run's own allocator
starts at 1 and collides. Latent — the runtime never takes that path.
- The four scenarios that slot ids turned green say so, instead of describing
themselves as open reproductions in the CI summary table.
- `appendUniqueEvents`'s test no longer claims a sorted order is one no load
returns; under slot ids it is exactly what a load returns.
- Adds `@workflow/world-vercel` to the capability changeset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@msullivanmsullivanAwaiting requested review from msullivan
@fantixfantixAwaiting requested review from fantix

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

2 participants

@karthikscale3@VaguelySerious