Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); implement event-sourced architecture by pranaygp · Pull Request #621 · vercel/workflow · GitHub
Skip to content

implement event-sourced architecture - #621

Merged
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events
Jan 23, 2026
Merged

implement event-sourced architecture#621
pranaygp merged 39 commits into
mainfrom
pranaygp/perf-phase-3b-atomic-events

Conversation

@pranaygp

@pranaygppranaygp commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

Pranay:

corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154

important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version of workflow will be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).

The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)

I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):

Docs preview

small: I also removed the unused run paused/resumed stuff which we've never used to simplify

Summary

Implement event-sourced architecture for runs, steps, and hooks:

  • Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
  • Add step_retrying event for non-fatal step failures that will be retried
  • Remove fatal field from step_failed event (step_failed now implies terminal failure)
  • Rename step's lastKnownError to error for consistency with server
  • Update world-local, world-postgres, and world-vercel to create/update entities from events via events.create()
  • Entities (runs, steps, hooks) are now materializations of the event log
  • Fix hook token conflict error to use WorkflowAPIError with status 409
  • Move event log corruption check to step_created event for earlier detection
  • BREAKING CHANGE: Remove unused run_paused/run_resumed events and paused status

This makes the system faster, easier to reason about, and resilient to data inconsistencies.

Test plan

  • TypeScript compiles
  • Unit tests pass
  • E2E tests pass

🤖 Generated with Claude Code

@changeset-bot

changeset-botBot commented Dec 16, 2025

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b7a352a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 18 packages
NameType
@workflow/worldMinor
@workflow/world-localPatch
@workflow/world-postgresPatch
@workflow/errorsPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/world-vercelPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/docs-typecheckPatch
workflowPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/astroPatch
@workflow/sveltekitPatch
@workflow/aiPatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

@github-actions

github-actionsBot commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production457038495
✅ 💻 Local Development418032450
✅ 📦 Local Production418032450
✅ 🐘 Local Postgres418032450
✅ 🪟 Windows450045
❌ 🌍 Community Worlds311610192
Total17871611342082

❌ Failed Tests

🌍 Community Worlds (161 failed)

mongodb (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

redis (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

starter (41 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

turso (40 failed):

  • addTenWorkflow
  • addTenWorkflow
  • should work with react rendering in step
  • promiseAllWorkflow
  • promiseRaceWorkflow
  • promiseAnyWorkflow
  • readableStreamWorkflow
  • hookWorkflow
  • webhookWorkflow
  • sleepingWorkflow
  • nullByteWorkflow
  • workflowAndStepMetadataWorkflow
  • outputStreamWorkflow
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions
  • fetchWorkflow
  • promiseRaceStressTestWorkflow
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • closureVariableWorkflow - nested step functions with closure variables
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly
  • Calculator.calculate - static workflow method using static step methods from another class
  • AllInOneService.processNumber - static workflow method using sibling static step methods
  • ChainableService.processWithThis - static step methods using this to reference the class
  • thisSerializationWorkflow - step function invoked with .call() and .apply()
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro4104
✅ example4104
✅ express4104
✅ fastify4104
✅ hono4104
✅ nextjs-turbopack4401
✅ nextjs-webpack4401
✅ nitro4104
✅ nuxt4104
✅ sveltekit4104
✅ vite4104
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable4104
✅ express-stable4104
✅ fastify-stable4104
✅ hono-stable4104
✅ nextjs-turbopack-stable4500
✅ nextjs-webpack-stable4500
✅ nitro-stable4104
✅ nuxt-stable4104
✅ sveltekit-stable4104
✅ vite-stable4104
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack4500
❌ 🌍 Community Worlds
AppPassedFailedSkipped
✅ mongodb-dev300
❌ mongodb5400
✅ redis-dev300
❌ redis5400
✅ starter-dev300
❌ starter4410
✅ turso-dev300
❌ turso5400

📋 View full workflow run

@pranaygpGraphite App

pranaygp commented Dec 16, 2025

Copy link
Copy Markdown
ContributorAuthor

@pranaygp
pranaygpforce-pushed the pranaygp/perf-phase-3b-atomic-events branch from 6ebd4c5 to 2e46b8aCompareDecember 16, 2025 05:45
@pranaygp
pranaygpforce-pushed the pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency branch from eece359 to 290e879CompareDecember 16, 2025 05:45

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.

Key Changes

  • Added events.createBatch() method to the World interface for creating multiple events in a single operation
  • Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
  • Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file
FileDescription
packages/world/src/interfaces.tsAdded createBatch() method signature with JSDoc documentation to the Storage events interface
packages/world-vercel/src/storage.tsIntegrated batch event creation into the storage adapter
packages/world-vercel/src/events.tsImplemented createWorkflowRunEventBatch() using parallel API calls via Promise.all
packages/world-postgres/src/storage.tsImplemented batch creation using a single INSERT query with multiple values for optimal database performance
packages/world-local/src/storage.tsImplemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage
packages/core/src/runtime.tsRefactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity
.changeset/brave-dots-bake.mdAdded changeset documenting the performance improvement across all affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-local/src/storage.ts Outdated
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/core/src/runtime.ts
Comment threadpackages/world-vercel/src/events.ts Outdated
Comment threadpackages/world-postgres/src/storage.ts Outdated
pranaygpand others added 19 commits January 22, 2026 12:58
- Fix broken link in hook-conflict.mdx (/docs/foundations/webhooks -> /docs/api-reference/workflow/create-webhook)
- Add hook-conflict to errors index page so it's discoverable by the docs link validator
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update tests to expect hook_conflict events instead of thrown errors when
duplicate hook tokens are used. This aligns with the new event-sourced
approach where conflicts are recorded as events rather than thrown.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion property to World interface to track world package version
- Add specVersion to WorkflowRun schema and run_created event data
- World implementations (vercel, local, postgres) set specVersion from npm version
- Server can use specVersion to route operations based on world version
- Add specVersion display to observability UI attribute panel
- Add spec_version column to postgres runs schema
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Properly generates migration with drizzle-kit CLI
- Removes deprecated 'paused' status from enum
- Adds spec_version column
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add RunNotSupportedError for runs requiring newer world versions
- Add semver-based version utilities (isLegacyVersion) to @workflow/world
- World implementations check specVersion and route to legacy handlers
- Legacy runs (< 4.1.0): run_cancelled skips event storage, wait_completed stores event only
- New runs always get current world version (4.1.0-beta.0)
- Make EventResult.event optional for legacy compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace semver-based version compatibility with explicit integer spec versions:
- SPEC_VERSION_LEGACY (1): pre-event-sourcing runs
- SPEC_VERSION_CURRENT (2): event-sourced architecture
Use branded SpecVersion type to enforce importing from @workflow/world.
Remove semver dependency from world, world-local, and world-postgres.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add error_cbor bytea columns to workflow_runs and workflow_steps tables
- Deprecate text error column, rename to errorJson with fallback parsing
- Remove JSON.stringify from error writes (run_failed, step_failed, step_retrying)
- Add parseErrorJson helper for backwards compatibility with legacy data
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace generic Error with WorkflowRuntimeError for runtime assertions
- Add explicit check for run entity in run_created response
- Use run.runId instead of event.runId for consistency
- Use actual run status instead of hardcoded 'pending' in attributes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion field to Step, Hook, and Event interfaces in @workflow/world
- Add spec_version column to steps, hooks, events tables in postgres schema
- Set specVersion to SPEC_VERSION_CURRENT when creating entities in all worlds
- Update migration to include spec_version columns for all entity tables
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split storage.ts (1041 lines) into smaller, focused modules:
- storage/filters.ts: Data filtering helpers
- storage/helpers.ts: ULID and date utilities
- storage/hooks-storage.ts: Hook CRUD operations
- storage/legacy.ts: Legacy event handling
- storage/runs-storage.ts: Run get/list operations
- storage/steps-storage.ts: Step get/list operations
- storage/events-storage.ts: Event create/list operations
- storage/index.ts: Main composition
Also extracted test helpers to test-helpers.ts for reusability.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The World.specVersion string property was never actually read - only
the numeric SPEC_VERSION_CURRENT is used for backwards compatibility.
- Remove genversion dependency and generated version.ts from @workflow/world
- Remove specVersion property from World interface and all implementations
- Minor fix: correct error message to reference 'workflow' package
- Minor fix: correct error source priority in world-vercel events.ts
- Minor fix: update comment in runtime.ts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
These packages no longer need genversion since we removed the
World.specVersion property in the previous commit.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No longer needed after removing genversion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for world-local and world-postgres covering:
- Legacy runs (specVersion < 2 or null/undefined)
- run_cancelled handling (updates run, no event stored)
- wait_completed handling (stores event only)
- Rejection of unsupported events
- Hook cleanup on cancellation
- Future runs (specVersion > current)
- Rejection with RunNotSupportedError
- Current version runs (normal processing)
- Legacy error parsing (errorJson field parsing)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When resumeHook() is called on a legacy run (specVersion < 2), the
hook_received event was previously rejected. This adds support for
storing hook_received events on legacy runs without entity mutation,
matching the behavior of wait_completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
…tarted events
Replace with run_completed, run_failed, and run_started equivalents.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@@ -0,0 +1,12 @@
---
"@workflow/world": minor

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this should cause a minor bump transitive to everything that depends on the world package

pranaygpand others added 6 commits January 22, 2026 14:07
The manually-created EventWithRefsSchema was missing the specVersion field,
which caused specVersion to be stripped when using lazy (refs) mode for events.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Core runtime now sends specVersion in run_created eventData
- world-local accepts specVersion from eventData (defaults to current)
- world-postgres accepts specVersion from eventData (defaults to current)
This matches workflow-server behavior where v2 endpoints accept
specVersion from the client, while v1 endpoints default to legacy.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add specVersion to BaseEventSchema (event level, not eventData)
- Remove specVersion from RunCreatedEventSchema.eventData
- Core runtime sends specVersion on event object
- world-local reads specVersion from event, propagates to run/step/hook entities
- world-postgres reads specVersion from event, propagates to run/step/hook entities
This ensures specVersion flows from client through event to all created entities.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- specVersion is optional in all entity schemas (runs, steps, hooks, events)
for backwards compatibility with legacy data in storage
- Runtime always sends specVersion on event requests
- world-local and world-postgres provide fallback to SPEC_VERSION_CURRENT
- Test helpers include specVersion in all event creation calls
- EventWithRefsSchema in world-vercel defaults specVersion to 1 for legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two tests were calling queue.queue() without setting up
VERCEL_DEPLOYMENT_ID, causing them to fail with "No deploymentId
provided" error before reaching the code they were testing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The server's CreateEventSchemaV2 requires specVersion on all events,
but only run_created was sending it. This caused 400 Bad Request errors
for all other event types (run_started, run_completed, run_failed,
run_cancelled, step_created, step_started, step_completed, step_failed,
step_retrying, hook_created, hook_received, wait_created, wait_completed).
Now all event creation calls include specVersion: SPEC_VERSION_CURRENT.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@pranaygp@TooTallNate@VaguelySerious