Uh oh!
There was an error while loading. Please reload this page.
perf(core): initialize lazy hook replay from hook_received stream - #3345
Conversation
🦋 Changeset detectedLatest commit: 9ce3581 The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed E2E Test SummarySummary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 128734ms → this run 146913ms (Δ +18179ms, +14%) 1020 steps (queue-hop) Cumulative STSO time: 3198ms over 1 samples No 📜 Previous results (2)2daae33Wed, 05 Aug 2026 18:48:38 GMT · run logs
549d6f0Wed, 05 Aug 2026 01:54:01 GMT · run logs
ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
On a lazy hook queue delivery, the consumer's idempotent hook_received re-ensure is hoisted above run_started and doubles as the invocation's setup request: it asks the World to return the current replay log with the write (new advisory CreateEventParams.preloadEvents), so one HTTP request yields the canonical event, the reconstructed run, and the complete replay log — skipping both the run_started POST and the initial events.list. - world: optional `preloadEvents?: true` on CreateEventParams, the hook_received dual of skipPreload; Worlds may ignore it - world-vercel: createHookReceivedPreloadEventV4 sends the frame Accept on eligible hook_received posts and decodes either response mode — frames via the response decoder extracted from the LIST consumer (GET behavior unchanged), CBOR via the shared materialized-response mapping. The run is reconstructed from run_created/run_started (plus attr_set folds), the canonical event found by x-wf-event-id, and resumeId now survives frame decoding so the runtime can match it - core: new fast path before the generic run-state setup, guarded on hookInput.resumeId + payloadDigest; a validated COMPLETE preload (hasMore false — this path has no cursor-continuation machinery) initializes workflowRun/preloadedEvents/maxEventsLimit directly, anything else falls back to the run_started setup without re-posting the hook; error classification matches the existing re-ensure (terminal → consume, transient → redeliver); setup source reported via workflow.resume_setup_source (never workflow.hook.resilient_resume_materialized, which stays a recovery-only signal) - producer resumeHook() is unchanged and never sets preloadEvents Based directly on main (no dependency on #3124/#3191); pairs with workflow-server's streamed hook_received replay-log response, which deploys first — the SDK negotiates per request and falls back safely against older servers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
549d6f0 to
2daae33Compare
TooTallNate
left a comment
There was a problem hiding this comment.
Reviewed at 549d6f0. Locally: build + typecheck green, core 1920 passed / 3 expected fail, world-vercel 339 — with the 12 consumer-preload tests covering every fallback and error branch I went looking for.
What held up under scrutiny:
- The completeness validation is the right shape: run +
startedAt, non-empty events, non-null cursor,hasMore === false, numericmaxEvents(this response plays run_started's role, so a missing ceiling would silently disable event-limit enforcement — good catch), both lifecycle events, and thehook_receivedmatching thisresumeId. On the cursor requirement: I confirmed the backend synthesizes a cursor on the final page even for single-page logs, so the check doesn't dead-letter short runs — but that server behavior is now load-bearing for this fast path; a code comment noting the dependency would help the next person. - Error classification is byte-for-byte consistent with the existing re-ensure: HookNotFound/RunExpired consume the delivery; everything else (EntityConflict, truncated stream, transport) rethrows for redelivery and the
(runId, resumeId)claim converges. And the deliberate omission ofHookResilientResumeMaterialized(this path carries no recovery signal) keeps that metric honest. - The terminal-event check before engine dispatch correctly plugs the QuickJS gap (it dispatches before the node loop's terminal detection), and
preloadedEventsCompleteas an explicit attestation — rather than widening the first-invocation heuristic — is the safer design. resumeIdthreading throughbuildEventFromV4is essential and easy to miss: without it, frame-decoded events would silently fail the matching check and the fast path would never fire. The comment says exactly that.remoteRefBehavior: 'resolve'override on the preload request is right (v4 has no refs endpoint to hydrate lazy descriptors mid-replay), andreconstructRunFromReplayEventscarries every field downstream consumers read — includingdeploymentIdandspecVersion, which the in-flight deployment-affinity and slot-identity work key off.
Three asks before merge:
- Rebase — the PR is currently CONFLICTING with main (a one-file test conflict in
events.test.tsvs #3334). - Coordinate with #2960 (deployment-affinity guard, also open). Its design places the guard ahead of the lazy-hook re-ensure with the explicit invariant "a misrouted resume writes nothing here" — this fast path hoists the
hook_receivedwrite above where that guard will sit. I believe the combination is still safe (the write is idempotent, involves no key derivation, and the guard still precedes any replay/step execution), but whichever PR lands second must reconcile the placement and rewrite that comment — the "writes nothing" invariant will no longer be literally true, and it should be weakened deliberately rather than silently. - Changeset bump:
@workflow/worldgains a new public interface field (preloadEvents) plus documentedEventResultsemantics — per the convention we've applied on recent PRs, new API surface on the world interface should beminor, notpatch.
For the record, the paired backend PR's red trigger lane ran with main's SDK (no Accept header → this feature dormant), and its failures match the varied preview-lane flakiness other backend branches see — not this pair. The real proof of the active path will be this PR's own e2e once rebased, since the backend half is already deployable ahead.
Nice perf win with a genuinely safe fallback story. Approving.
karthikscale3
commented
Aug 6, 2026
CI run completed without any issues (Except one unrelated flake). Checked datadog and compared CI run span metrics pre server side merge vs post server side merge: The p50 improvement appears across every framework cohort: SvelteKit: 1,194 → 939 ms, −255 ms Merging this now as all review comments are addressed and the confidence levels are high. |
Uh oh!
There was an error while loading. Please reload this page.
No backport to This is a latency optimization that adds new API surface — an advisory To override, re-run the Backport to stable workflow manually via |
TooTallNate
commented
Aug 6, 2026
Heads-up: the #2960 ↔ #3345 ordering interaction flagged in my review above is now a red unit test on
As discussed in the review, I believe the behavior is still safe — the write is idempotent, involves no key derivation, and the guard still runs before any replay or step execution — but #2960's "a misrouted resume writes nothing here" invariant is now false and its test enforces it, so
(Found while reviewing #3372, whose author independently hit the same failing test and correctly identified it as pre-existing.) |
…#3374) The lazy hook fast path (#3345) hoisted the consumer's hook_received write above the deployment-affinity guard (#2960), so a misrouted lazy resume wrote its event before the guard could re-route the delivery. Stamp the run's pinned deployment on the resume message (hookInput.deploymentId, from the producer's resume context) and, on the consumer, compare it against the ambient deployment id immediately before the fast path: a match continues with no run fetch, a mismatch fetches the authoritative run and hands it to the existing guard — which keeps sole ownership of re-route/fail policy and remains the authoritative protection before replay and step execution. The re-routed message preserves the complete hookInput (it may hold the only copy of the resume payload). Older messages without the field, and worlds without deployment affinity, are unchanged: they skip the pre-check and rely on the authoritative guard, the pre-guard write staying convergent per (runId, resumeId). Fixes the misrouted-lazy-resume unit test broken by the #2960/#3345 ordering: a modern misrouted resume now re-routes with zero event writes, asserted for both hook_received and run_started. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
What
On a lazy hook queue delivery, the consumer's idempotent
hook_receivedre-ensure is hoisted aboverun_startedand asks the World to return the current replay log with the write (new advisoryCreateEventParams.preloadEvents). When a complete preload comes back, the invocation initializes replay from that one request and skips both therun_startedPOST and the initialevents.list; otherwise it falls back to the existing setup without re-posting the hook.Why
Each removed round trip sits directly on hook-resume latency (queue receipt → replay start). Folding the re-ensure and the replay load into one request cuts consumer startup/TTFS by roughly two request latencies on the normal lazy-resume path.
Notes
preloadEvents; a preload is trusted only when validated as complete (run +hasMore: false+maxEvents+ lifecycle events + the matchingresumeId). A terminal event in the preload consumes the delivery before engine dispatch.resumeHook()is unchanged and never setspreloadEvents.workflow.resume_setup_source(hook_received_stream|hook_received_fallback).🤖 Generated with Claude Code