Skip to content

perf(world-local): index hook lookups instead of scanning the global event log - #2830

Merged
AndrewBarba merged 4 commits into
mainfrom
perf/world-local-hook-indexes
Jul 8, 2026
Merged

perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom
perf/world-local-hook-indexes

Conversation

@AndrewBarba

@AndrewBarbaAndrewBarba commented Jul 8, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).

Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:

  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.

Changes

@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):

  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.

@workflow/world — reenqueueActiveRuns uses a page size of 200.

Benchmark

Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):

turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7

Flat instead of linear, and ~3x faster even on a cold data dir.

Testing

  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.

@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba requested review from a team and ijjk as code ownersJuly 8, 2026 20:26
@changeset-bot

changeset-botBot commented Jul 8, 2026•
edited
Loading

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 86021d9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Jul 8, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm

@github-actions

github-actionsBot commented Jul 8, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

No benchmark result files found in benchmark-results

@github-actions

github-actionsBot commented Jul 8, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

✅ All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious approved these changes Jul 8, 2026
View reviewed changes

@VaguelySeriousVaguelySerious left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass

@AndrewBarba
AndrewBarba enabled auto-merge (squash) July 8, 2026 21:10
@AndrewBarba
AndrewBarba disabled auto-merge July 8, 2026 21:11
@AndrewBarba
AndrewBarba enabled auto-merge (squash) July 8, 2026 21:43
@AndrewBarba
AndrewBarba merged commit 79a9813 into mainJul 8, 2026
170 of 181 checks passed
@github-actionsgithub-actionsBot mentioned this pull request Jul 8, 2026
Merged
@github-actions

github-actionsBot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)

@github-actionsgithub-actionsBot mentioned this pull request Jul 8, 2026
Closed
@VaguelySeriousVaguelySerious mentioned this pull request Jul 9, 2026
Merged
4 tasks
This was referenced Jul 9, 2026
Open
Closed
Open
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
perf(world-local): index hook lookups instead of scanning the global event log by AndrewBarba · Pull Request #2830 · vercel/workflow · GitHub
Skip to content
perf(world-local): index hook lookups instead of scanning the global event log - #2830
Merged
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Jul 8, 2026
Merged
perf(world-local): index hook lookups instead of scanning the global event log#2830
AndrewBarba merged 4 commits into
mainfrom 
perf/world-local-hook-indexes
Conversation
@AndrewBarba
@AndrewBarbaAndrewBarba
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Summary
The eve agent-workflow-stress CI job (run) showed sequential turns against world-local getting linearly slower — turn 2: 1.66s → turn 100: 4.58s (~+30ms/turn).
Root cause: several world-local hot paths do O(total history) work on every operation, dominated by hooks:
  1. Every first-time hook_created hit the token-claim recovery path (events-storage.ts claim loop → rebuildLiveHookByTokenFromEventLog), which read and zod-parsed every event file ever written across all runs to prove the token had no live prior hook. Events are append-only and never deleted, so this grew forever.
  2. hooks.get / getByToken fallbacks did the same full-log scan for any hook whose entity was deleted (all hooks of completed runs).
  3. Every run termination (run_completed / run_failed / run_cancelled → deleteAllHooksForRun) read every live hook entity in the world to match runId.
  4. Startup recovery paged runs.list with the default page size of 20, costing ~O(N²/20) directory scans over accumulated run history.
Changes
@workflow/world-local — new durable hook indexes (src/storage/hook-index.ts):
  • hooks/token-index/{sha256(token)}/{eventId}.json and hooks/id-index/{hookId}/{eventId}.json mirror every persisted hook_created event. Rebuild/uniqueness paths now read O(events for that key) instead of O(all events ever). Entries are written before the event publish, so a crash can only leave a dangling entry (skipped by readers), never a committed event the index can't see.
  • hooks/by-run/{runId}-{hookId}.json markers let run-termination cleanup find a run's hooks with a prefix readdir instead of reading every live hook. Markers are written before the entity and reaped on dispose/termination.
  • The liveness semantics of the old full scan are preserved: its in-log closure replay (hook_disposed / terminal-run events) is strictly implied by the existing dispose-lock and terminal-run-status checks, because both are durably written before their events are appended and are never deleted.
  • Legacy data dirs: ensureHookIndexes performs a one-time backfill (a single scan — the same cost as ONE pre-index rebuild) and publishes a completion marker.
  • getByToken resolves through the token claim file (O(1)) before falling back to the entity scan.
  • paginatedFileSystemQuery reads files with bounded concurrency (32) instead of strictly sequentially, and getObjectCreatedAt compiles its regex once per query instead of once per filename.
@workflow/world — reenqueueActiveRuns uses a page size of 200.
Benchmark
Micro-benchmark mirroring the eve stress shape (each turn: create+start a turn run, create+dispose a hook on a long-lived session run, 5 steps × 3 events, complete the turn run):
turnsmain (ms/turn)this PR (ms/turn)
1–2571.924.4
76–100175.721.8
176–200393.423.7
Flat instead of linear, and ~3x faster even on a cold data dir.
Testing
  • packages/world-local: full suite passes (450 tests), including all existing hook crash-recovery / claim-convergence / cross-instance tests.
  • New src/storage/hook-index.test.ts: index maintenance, marker reaping on dispose, conflict detection after crash-lost claims, per-run cleanup isolation, claim fast path, legacy-backfill scenarios (rebuild, conflicts, cleanup), and a perf guard asserting hook creation doesn't scale with unrelated event history.
  • packages/world and packages/world-postgres unit tests pass (recovery page size).
  • @workflow/world-testing embedded conformance suite couldn't run locally (no Rust toolchain for the SWC plugin build) — relying on CI.
@AndrewBarba
perf(world-local): index hook lookups instead of scanning the global …
03ccf74
…event log
Hook creation, hook cache rebuilds, and token lookups previously read
and parsed every event file ever written across all runs, and run
termination read every live hook entity — making every turn of a
long-lived local session slower as history accumulated. Maintain
durable per-token / per-hookId indexes and per-run hook markers
(with a one-time backfill for pre-index data dirs), resolve tokens
through the claim file, read directory listings concurrently, reuse
compiled filename regexes, and raise the recovery page size.
@AndrewBarba
AndrewBarba
requested review from
a team and ijjk
as code ownersJuly 8, 2026 20:26
@changeset-bot
changeset-botBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

🦋 Changeset detected
Latest commit: 86021d9
The changes in this PR will be included in the next version bump.
This PR includes changesets to release 18 packages
NameType
@workflow/world-localPatch
@workflow/buildersPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/vitestPatch
@workflow/webPatch
@workflow/world-postgresPatch
@workflow/world-testingPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/nuxtPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
workflowPatch
@workflow/web-sharedPatch
Not sure what this means? Click here to learn what changesets are.
Click here if you're a maintainer who wants to add another changeset to this PR
@vercel
vercelBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
The latest updates on your projects. Learn more about Vercel for GitHub.
ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-nextjs-workflow-webpackReadyReadyPreview, CommentJul 8, 2026 9:07pm
example-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-astro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-express-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-fastify-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-hono-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nitro-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-nuxt-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-sveltekit-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-tanstack-start-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workbench-vite-workflowReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-docsReadyReadyPreview, Comment, Open in v0Jul 8, 2026 9:07pm
workflow-swc-playgroundReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-tarballsReadyReadyPreview, CommentJul 8, 2026 9:07pm
workflow-webReadyReadyPreview, CommentJul 8, 2026 9:07pm
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
No benchmark result files found in benchmark-results
@github-actions
github-actionsBot
commented
Jul 8, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
🧪 E2E Test Results
✅ All tests passed
Summary
PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415
Details by Category
✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027
📋 View full workflow run
@AndrewBarba
review: keep world package untouched, batch backfill reads
86021d9
- Revert the recovery page-size change in @workflow/world; default
world-local runs.list to a 200-item page instead
- Read backfill event/hook files with bounded concurrency (32),
matching paginatedFileSystemQuery, to cut one-time migration cost
on large legacy data dirs (slow-fs platforms especially)
VaguelySerious
VaguelySerious
approved these changes
Jul 8, 2026
View reviewed changes
@VaguelySeriousVaguelySerious
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming all tests pass
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:10
@AndrewBarba
AndrewBarba
disabled auto-merge
July 8, 2026 21:11
@AndrewBarba
AndrewBarba
enabled auto-merge (squash)
July 8, 2026 21:43
@AndrewBarba
AndrewBarba
merged commit 79a9813
into
mainJul 8, 2026

170 of 181 checks passed

@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Merged

@github-actions
github-actionsBot
commented
Jul 8, 2026

Copy link


Copy Markdown

Contributor
Backport PR opened against stable: #2839. Merge conflicts were resolved by AI — please review carefully. (backport job run)
@github-actionsgithub-actionsBot
mentioned this pull request
Jul 8, 2026
 Closed

@VaguelySeriousVaguelySerious
mentioned this pull request
Jul 9, 2026
 Merged

4 tasks

This was referenced Jul 9, 2026
 Open

 Closed

 Open

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@VaguelySeriousVaguelySeriousVaguelySerious approved these changes
@karthikscale3karthikscale3karthikscale3 approved these changes
@ijjkijjkAwaiting requested review from ijjk

Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

3 participants

@AndrewBarba@VaguelySerious@karthikscale3