fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

fix(world-vercel): hold the ws channel registry on globalThis - #3699

Closed
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry
Closed

fix(world-vercel): hold the ws channel registry on globalThis#3699
shalabhc wants to merge 1 commit into
mainfrom
fix-ws-transport-global-registry

Conversation

@shalabhc

@shalabhcshalabhc commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Due to bundling the world-vercel (#3493), we can have two copies of the module which cause websockets to not work from durabench.

Note in SDK use from e2e tests I have seen websockets work just fine. It breaks on durabench because of it's instrumentation initialization.

This PR fixes the issue by moving const transports out of module state into globalThis.

Details ## What

The WS events transport's channel registry moves from module scope onto a
Symbol.for() key on globalThis — the same technique @workflow/core already
uses for the World cache — along with the two once-per-process log latches.

Why

The transport's open and its lookup are reached through two different World
instances
:

  • openWsChannel runs inside world-vercel's createQueueHandler, and the
    runtime builds that consumer from getWorldHandlers();
  • a write reaches resolveWsTransport (a lookup, never a create) through
    getWorld().

Those are two createWorld() calls, cached under two different globalThis
symbols in core. The only thing that made them meet was sharing one module
instance
of this package, because const transports was module-scope.

#3493 bundled @workflow/world-vercel into the Next.js server output instead of
leaving it in serverExternalPackages. Next then gives the instrumentation.ts
entry its own copy of the module, separate from the route entries. So in an
app whose register() warms the world:

// instrumentation.tsconst{ getWorld }=awaitimport("workflow/runtime");awaitgetWorld();// events-writing World, built in the instrumentation copy

…the flow route's later getWorldHandlers() builds the consumer's World in the
route copy. Opens land in one Map; every lookup reads the other, empty one.
The socket is opened and carries nothing — every event silently falls back to
HTTP, deterministically, for the life of the process. That is what
vercel-labs/durabench's apps/workflow-host hit: sockets in the trace, zero
event frames on them.

Nothing about the WS design was wrong, and this is not a reason to revert #3493
(its cold-start win is real). A module-scope singleton in a package that a
bundler may duplicate is the bug.

Verification

Notes for review

  • The symbol carries a /v1 suffix so a future change to WsEventsTransport's
    shape gets a fresh registry rather than a structurally-incompatible hit from an
    older copy sharing the process. Cross-copy use is structural (request,
    release, close); there is no instanceof on the transport.
  • Deliberately not in this PR, and worth doing separately: a debug breadcrumb
    when resolveWsTransport misses (the silence here cost several
    investigations); a transport assertion in the e2e-vercel-ws-transport lane,
    which today passes whether events go over WS or fall back; and re-combining
    getWorld/getWorldHandlers, which the code comment in
    packages/core/src/runtime/world.ts already contemplates and which is what
    turned "duplicated module" into a systematic miss rather than a race.
  • The workbench app that lane deploys only calls registerOTel in its
    instrumentation, which is why CI never saw this; adding a world warm-up there
    would give the lane the failing shape.

@changeset-bot

changeset-botBot commented Aug 20, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3d940b8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
NameType
@workflow/world-vercelPatch
@workflow/cliPatch
@workflow/corePatch
@workflow/webPatch
workflowPatch
@workflow/world-testingPatch
@workflow/buildersPatch
@workflow/nextPatch
@workflow/nitroPatch
@workflow/vitestPatch
@workflow/web-sharedPatch
@workflow/astroPatch
@workflow/nestPatch
@workflow/rollupPatch
@workflow/sveltekitPatch
@workflow/vitePatch
@workflow/nuxtPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercelBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated (UTC)
example-nextjs-workflow-turbopackReadyReadyPreviewAug 20, 2026 8:55pm
example-nextjs-workflow-webpackReadyReadyPreviewAug 20, 2026 8:55pm
example-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-astro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-express-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-fastify-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-hono-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nestjs-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nitro-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-nuxt-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-python-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-sveltekit-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-tanstack-start-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workbench-vite-workflowReadyReadyPreviewAug 20, 2026 8:55pm
workflow-docsReadyReadyPreview, v0Aug 20, 2026 8:55pm
workflow-swc-playgroundReadyReadyPreviewAug 20, 2026 8:55pm
workflow-tarballsReadyReadyPreviewAug 20, 2026 8:55pm
workflow-webReadyReadyPreviewAug 20, 2026 8:55pm

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

▲ Vercel Production (235 failed)

astro-node (6 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

astro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

example-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

example-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

express-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

express-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

fastify-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

fastify-quickjs (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

hono-node (8 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

hono-quickjs (7 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nest-node (12 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nest-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-turbopack-node (4 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

nextjs-turbopack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nextjs-webpack-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nextjs-webpack-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

nitro-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting

nuxt-node (11 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

nuxt-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

sveltekit-node (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes workflow throws after awaited setAttributes: attribute still persists on the failed run

sveltekit-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

tanstack-start-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

tanstack-start-quickjs (9 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

vite-node (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: initial attributes are seeded on run creation
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands

vite-quickjs (10 failed):

  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_41M0GFJPPC0GYXBK33KRXND1AA | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_41M0GFJF9Q0GT52K3YPD2X0SEN | 🔍 observability
  • hookGetConflictWorkflow - hook.getConflict() resolves with the conflicting run when token is already registered | wrun_41M0GFKTSP0GG7FBCDQ0RMBP81 | 🔍 observability
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable | wrun_41M0GFNFHC0GTK4YT3SQPJR3Z8 | 🔍 observability
  • setAttributes start: reserved-prefix initial attributes are seeded with allowReservedAttributes
  • setAttributes setAttributesWorkflow: workflow-body calls append native attr_set events and merge correctly
  • setAttributes setAttributesInsideStepWorkflow: step-body calls append attributed native events
  • setAttributes fire-and-forget: void setAttributes lands without awaiting
  • setAttributes Promise.all of disjoint-key writes: every key lands
  • setAttributes validation DX: invalid writes throw catchable FatalErrors naming rule and limit

💻 Local Development (4 failed)

nextjs-webpack-canary-node (1 failed):

  • runClassSerializationWorkflow - Run instances serialize across workflow/step boundaries | wrun_41M0GFPC0R0GGA9CR0D2NTZEYC

nextjs-webpack-stable-node (2 failed):

  • pages router addTenWorkflow via pages router
  • pages router sleepingWorkflow via pages router

nextjs-webpack-stable-quickjs (1 failed):

  • health check (CLI) - workflow health command reports healthy endpoints

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

28 flaky tests
  • abortAnyInStepWorkflow: AbortSignal.any inside a step composes deserialized signals (nextjs-webpack)
  • abortParallelWorkflow: abort cancels all parallel steps (nuxt)
  • abortSurvivesReplayWorkflow: controller state consistent across replay (nextjs-webpack)
  • abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries (nextjs-webpack)
  • addTenWorkflow via pages router (nextjs-webpack)
  • cancelRun via CLI - cancelling a running workflow (nextjs-turbopack)
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously (nextjs-turbopack)
  • hookClaimOnlyMutexWorkflow - hook works as a pure run mutex without payload data (nextjs-webpack)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (express)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (hono)
  • hookMinRetentionWorkflow - terminal Hook cannot resume and its token stays unavailable (nuxt)
  • Promise.all of disjoint-key writes: every key lands (hono)
  • Promise.all of disjoint-key writes: every key lands (nest)
  • Promise.all of disjoint-key writes: every key lands (nitro)
  • Promise.all of disjoint-key writes: every key lands (sveltekit)
  • promiseAllWorkflow via pages router (nextjs-webpack)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (astro)
  • setAttributesInsideStepWorkflow: step-body calls append attributed native events (tanstack-start)
  • sleepingWorkflow via pages router (nextjs-webpack)
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step (nextjs-webpack)
  • start: initial attributes are seeded on run creation (nextjs-turbopack)
  • start: initial attributes are seeded on run creation (vite)
  • start: reserved-prefix initial attributes are seeded with allowReservedAttributes (nextjs-turbopack)
  • validation DX: invalid writes throw catchable FatalErrors naming rule and limit (nitro)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (example)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nest)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (nextjs-webpack)
  • workflow throws after awaited setAttributes: attribute still persists on the failed run (vite)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

  • cold-start-warmup · suite warmup (tanstack-start) · at 20:56:36Z · abandoned wrun_01M0GFBDSHMR8XK4WAAG801DVB
  • run-pickup-stall · AllInOneService.processNumber - static workflow method using sibling static step methods (express) · at 20:58:17Z · abandoned wrun_01M0GFEWSGS7YFVPZP7QNX26H2

E2E Test Summary

Summary
PassedFailedSkippedTotal
❌ ▲ Vercel Production33432357424320
❌ 💻 Local Development391845584480
✅ 📦 Local Production392205584480
✅ 🐘 Local Postgres392205584480
✅ 🪟 Windows32000320
✅ 🌐 Cross-language Conformance90132141
✅ vercel-multi-region270027
Total15461239254818248
Details by Category

❌ ▲ Vercel Production

AppPassedFailedSkipped
❌ astro-node126628
❌ astro-quickjs123928
❌ example-node123928
❌ example-quickjs123928
❌ express-node124828
❌ express-quickjs123928
❌ fastify-node1211128
❌ fastify-quickjs1211128
❌ hono-node124828
❌ hono-quickjs125728
❌ nest-node1201228
❌ nest-quickjs123928
❌ nextjs-turbopack-node15343
❌ nextjs-turbopack-quickjs14893
❌ nextjs-webpack-node147103
❌ nextjs-webpack-quickjs14893
❌ nitro-node123928
❌ nitro-quickjs123928
❌ nuxt-node1211128
❌ nuxt-quickjs123928
✅ python-node80152
❌ sveltekit-node14299
❌ sveltekit-quickjs14299
❌ tanstack-start-node1221028
❌ tanstack-start-quickjs123928
❌ vite-node1221028
❌ vite-quickjs1221028

❌ 💻 Local Development

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
❌ nextjs-webpack-canary-node140119
✅ nextjs-webpack-canary-quickjs141019
❌ nextjs-webpack-stable-node15820
❌ nextjs-webpack-stable-quickjs15910
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 📦 Local Production

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🐘 Local Postgres

AppPassedFailedSkipped
✅ astro-stable-node134026
✅ astro-stable-quickjs134026
✅ express-stable-node134026
✅ express-stable-quickjs134026
✅ fastify-stable-node134026
✅ fastify-stable-quickjs134026
✅ hono-stable-node134026
✅ hono-stable-quickjs134026
✅ nest-stable-node134026
✅ nest-stable-quickjs134026
✅ nextjs-turbopack-canary-node141019
✅ nextjs-turbopack-canary-quickjs141019
✅ nextjs-turbopack-stable-node16000
✅ nextjs-turbopack-stable-quickjs16000
✅ nextjs-webpack-canary-node141019
✅ nextjs-webpack-canary-quickjs141019
✅ nextjs-webpack-stable-node16000
✅ nextjs-webpack-stable-quickjs16000
✅ nitro-stable-node134026
✅ nitro-stable-quickjs134026
✅ nuxt-stable-node134026
✅ nuxt-stable-quickjs134026
✅ sveltekit-stable-node15307
✅ sveltekit-stable-quickjs15307
✅ tanstack-start-node134026
✅ tanstack-start-quickjs134026
✅ vite-stable-node134026
✅ vite-stable-quickjs134026

✅ 🪟 Windows

AppPassedFailedSkipped
✅ nextjs-turbopack-node16000
✅ nextjs-turbopack-quickjs16000

✅ 🌐 Cross-language Conformance

AppPassedFailedSkipped
✅ python90132

✅ vercel-multi-region

AppPassedFailedSkipped
✅ nextjs-turbopack2700

📋 View full workflow run

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

The benchmark run for 3d940b8 failed. See the run logs for details.

commit 5e30b16 · Thu, 20 Aug 2026 19:57:57 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioBest (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstep1280 (+24%) 🔻1386 🔴 (+22%) 🔻1407 🔴 (+20%) 🔻1772 🔴 (+45%) 🔻30
TTFSstream1271 (+614%) 🔻1388 🔴 (+24%) 🔻1429 🔴 (+27%) 🔻1438 🔴 (+23%) 🔻30
TTFShook + stream511 (+2.8%)1677 🔴 (+16%) 🔻1755 🔴 (+19%) 🔻2073 🔴 (+34%) 🔻30
Fan-out TTFSPromise.all(100 steps)540 (+36%) 🔻868 (-3.0%)899 (-49%) 💚2203 (+17%) 🔻10
Fan-out TTLSPromise.all(100 steps)4730 (-1.2%)7893 (+28%) 🔻9735 (+55%) 🔻12654 (+76%) 🔻10
STSO1020 steps (inline)122 (+1.7%)169 (-32%) 💚192 (-36%) 💚353 (-30%) 💚1019
WO1020 steps165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚165458 (-30%) 💚1
CRTTfirst chunk (pooled)104 (+13%)166 (+26%) 🔻282 (+87%) 🔻3469 (+2002%) 🔻28

Streams

ScenarioCRTT 1stp75p90p99CDV maxiters
paced control (100/s, 60B)142 (+16%)152 (+5%)214 (-19%)310 (-72%)144 (-2%)10
size sweep (100/s, 160B-12KB)123 (-5%)149 (-33%)201 (-62%)473 (-39%)160 (-41%)10
replay gateway-gpt-5.4-nano-2000t (1x)151 (+25%)145 (-1%)190 (-26%)668 (-5%)263 (-49%)3
replay eve-gpt-5.6-sol-2000t (1x)245 (+92%)165 (+9%)228 (-11%)734 (-13%)363 (-28%)2
replay eve-gpt-5.6-sol-2000t (2x)326 (+167%)210 (±0%)351 (+9%)1334 (-60%)320 (-47%)3
ℹ️ Metric definitions & methodology

Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.

Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.

Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)

Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles

Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000teaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041

🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600

All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).

Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.

@github-actions

github-actionsBot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenariooutcomeeventsvirtreplayviolations
smoke-no-stepscompleted30msok0
smoke-one-stepcompleted60msok0
hook-at-step-startedcompleted120msok0
hook-at-step-completedcompleted120msok0
hook-at-hook-createdcompleted120msok0
deadline-hook-winscompleted71.0hok0
deadline-expirescompleted71.0hok0
long-sleepcompleted1130.0dok0
hook-never-arrivesstalled30msskipped0
step-retries-twicecompleted102.0sok0
parallel-stepscompleted90msok0
hook-on-execution-statecompleted120msok0
peek-hook-before-branchcompleted120msok0
peek-hook-after-branchcompleted120msok0
peek-hook-at-registrationcompleted120msok0
race-hook-before-probecompleted120msok0
race-hook-after-probecompleted120msok0
race-duplicate-deliverycompleted130msok0
attr-hook-before-stepcompleted110msok0
attr-hook-after-stepcompleted110msok0
attr-from-step-bodycompleted130msok0
fork-hook-after-timeoutcompleted141.0mok0
fork-hook-before-timeoutcompleted141.0mok0
count-hook-after-timeoutcompleted171.0mok0
count-hook-before-timeoutcompleted201.0mok0
stale-read-step-count-forkcompleted201.0mok0
stale-read-equal-step-countscompleted141.0mok0
step-vs-step-forkcompleted120msok0
step-vs-step-fork-fencedcompleted120msok0
fence-catches-benign-directioncompleted125msok0
in-flight-before-decisioncompleted171.0mok0
in-flight-before-decision-countedcompleted171.0mok0
in-flight-after-decisioncompleted192.0mok0
stale-read-step-count-fork-fencedcompleted201.0mok0
fork-hook-winscompleted131.0mok0
fork-timeout-winscompleted131.0mok0
unclaimed-payload-under-forkcompleted171.0mok0
claimed-payload-under-forkcompleted171.0mok0
writers-independent-step-bodiescompleted120msok0
writers-scripted-tempocompleted120msok0
cancel-mid-stepcancelled70msskipped0

Full trace: world-sim.txt

@shalabhc
shalabhc marked this pull request as ready for review August 20, 2026 20:50
@shalabhc
shalabhc requested a review from a team as a code ownerAugust 20, 2026 20:50
The WS events transport opens a channel from the queue consumer, which the
runtime builds from `getWorldHandlers()`, and looks it up on the write path
through `getWorld()` — two `createWorld()` calls cached under two different
`globalThis` symbols in @workflow/core. The registry that joins them was a
module-scope Map, so the handoff only worked while both calls ran in one module
instance of this package.
Since #3493 bundled the world into the Next.js server output instead of leaving
it external, that is no longer guaranteed: Next gives the instrumentation entry
its own copy of the module. An app whose `register()` warms the world
(`await getWorld()`) therefore creates the events-writing World in the
instrumentation copy and the queue consumer's World in the route entry, and
every lookup reads an empty Map — sockets open and carry nothing, silently, for
the life of the process.
Move the registry and the two once-per-process log latches onto a
`Symbol.for()` key on globalThis, the same technique core already uses for the
World cache, so the transport is correct however many copies a bundler makes.
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Co-Authored-By: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
Signed-off-by: Shalabh Chaturvedi <7066873+shalabhc@users.noreply.github.com>
'@workflow/world-vercel': patch
---

Hold the WebSocket events transport's channel registry on `globalThis` instead of at module scope, so an app that ends up with two copies of the bundled world in one process (for example a Next.js app whose `instrumentation.ts` warms the world) still resolves the channel its queue consumer opened instead of silently writing every event over HTTP.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too verboe

* registry at module scope, opens went into one Map and every write read the
* other, empty one: socket open, every event silently on HTTP.
*
* `/v1` so a later change to `WsEventsTransport`'s shape gets a fresh registry

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@pranaygppranaygp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect this is papering over a more global bug/regression from #3493 that's allowing duplicate instances of modules in the first place. maybe we can fix that instead of papering over the fix for websockets/transports specifically with this solution

@pranaygp

Copy link
Copy Markdown
Contributor

Closing in favour of #3728

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@shalabhc@pranaygp