[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail - #1947

Merged
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import
Jul 10, 2026
Merged

[rollup] Externalize optional @opentelemetry/api peer (only when absent) so framework builds don't fail#1947
VaguelySerious merged 9 commits into
mainfrom
peter/fix-otel-opaque-import

Conversation

@VaguelySerious

@VaguelySeriousVaguelySerious commented May 6, 2026

Copy link
Copy Markdown
Member

@opentelemetry/api is an optional peer dependency the SDK loads lazily inside a try/catch. When it isn't installed, SvelteKit's Rollup pipeline turns the unresolvable import('@opentelemetry/api') into a fatal build error:

[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"

This blocked upgrading SvelteKit apps (that don't install the peer) to workflow v5-beta.

Fix

In the shared @workflow/rollup transform plugin (used by SvelteKit, Nitro, Nuxt, Astro, and the Vite integration), externalize @opentelemetry/apionly when it can't be resolved:

  • Peer installed → resolve and bundle it normally. This matters for self-contained outputs (Nitro's .output/server, esbuild) that ship no node_modules — forcing it external there strands the runtime import and crashes the server with ERR_MODULE_NOT_FOUND.
  • Peer absent → mark it external so Rollup/Vite doesn't fail the build; the SDK's runtime try/catch then disables tracing.

Externalizing (rather than aliasing to an empty stub) is deliberate — a stub would permanently disable tracing even when the peer is present.

Why not make the import opaque?

An earlier revision built the specifier at runtime (['@opentelemetry','api'].join('/')) so bundlers couldn't follow it. That fixed the SvelteKit build but silently disabled tracing on esbuild/Nitro self-contained targets — the CLI's vercel-build-output-api build, Nitro, and Astro ship a bundle with no node_modules, so an opaque import can't be inlined and resolves to nothing at runtime (this included the example e2e deployment). Keeping the import static lets esbuild inline @opentelemetry/api and lets @vercel/nft trace it, so tracing keeps working across all targets.

Verification

  • Reproduced the original failure on Vite 6.4.1 and 7.3.6: the peer-absent SSR build fails with the exact Rollup failed to resolve import "@opentelemetry/api" error; the conditional external makes it pass. (Vite 8 no longer treats it as fatal, but externalizing is the version-robust fix.)
  • Confirmed the Nitro self-contained build (express workbench) bundles @opentelemetry/api into .output/server, and the server module evaluates with no node_modules present (no ERR_MODULE_NOT_FOUND). CI: all Nitro local lanes (express/fastify/hono/nitro/tanstack) green.
  • Confirmed the examplevercel-build-output-api bundle inlines @opentelemetry/api for the SDK's tracing path, so OTEL works on the BOA e2e lane. CI: sveltekit / nuxt / astro / vite / example Vercel-prod lanes green.
  • Added a @workflow/rollup regression test covering both paths (present → bundle, absent → external).
  • Pinned @opentelemetry/api directly in workbench/example (aligning it with the other workbenches) and documented the optional peer in the tracing guide.

Docs Preview

PageLink
Tracing (optional-peer note)https://workflow-docs-git-peter-fix-otel-opaque-import.vercel.sh/v5/docs/observability/tracing#enabling-tracing

`@opentelemetry/api` is an optional peer dependency loaded via a
try/catch'd `await import('@opentelemetry/api')`. Bundlers that
statically follow `import('…')` strings reject it as unresolvable when
the consumer hasn't installed it — SvelteKit's Rollup pipeline turns
this into a fatal build error:
[vite]: Rollup failed to resolve import "@opentelemetry/api" from
"src/routes/.well-known/workflow/v1/flow/__step_registrations.js"
Construct the specifier at runtime so the static analyzer can't resolve
it. The `globalThis`-cached symbol pattern used by `get-world-lazy.ts`
works for the same reason. Runtime semantics are unchanged: present →
loaded, absent → caught and tracing disabled.
Other build systems (Next.js, Nitro, Astro, Hono, Nuxt) tolerate the
unresolvable specifier today, so they were unaffected — but the same
pattern fails for any bundler that treats unresolved static specifiers
as fatal, and depending on transitive resolution to provide the peer
is brittle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercelBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

@changeset-bot

changeset-botBot commented May 6, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ec311ea

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
NameType
@workflow/rollupPatch
@workflow/buildersPatch
@workflow/astroPatch
@workflow/nitroPatch
@workflow/sveltekitPatch
@workflow/vitestPatch
workflowPatch
@workflow/cliPatch
@workflow/nestPatch
@workflow/nextPatch
@workflow/nuxtPatch
@workflow/vitePatch
@workflow/world-testingPatch
@workflow/corePatch
@workflow/web-sharedPatch
@workflow/webPatch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

All tests passed

Summary

PassedFailedSkippedTotal
✅ ▲ Vercel Production145302301683
✅ 💻 Local Development161702191836
✅ 📦 Local Production161702191836
✅ 🐘 Local Postgres161702191836
✅ 🪟 Windows15300153
✅ 📋 Other89401771071
Total7351010648415

Details by Category

✅ ▲ Vercel Production
AppPassedFailedSkipped
✅ astro126027
✅ example126027
✅ express126027
✅ fastify126027
✅ hono126027
✅ nextjs-turbopack15003
✅ nextjs-webpack15003
✅ nitro126027
✅ nuxt126027
✅ sveltekit14508
✅ vite126027
✅ 💻 Local Development
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 📦 Local Production
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🐘 Local Postgres
AppPassedFailedSkipped
✅ astro-stable128025
✅ express-stable128025
✅ fastify-stable128025
✅ hono-stable128025
✅ nextjs-turbopack-canary134019
✅ nextjs-turbopack-stable15300
✅ nextjs-webpack-canary134019
✅ nextjs-webpack-stable15300
✅ nitro-stable128025
✅ nuxt-stable128025
✅ sveltekit-stable14706
✅ vite-stable128025
✅ 🪟 Windows
AppPassedFailedSkipped
✅ nextjs-turbopack15300
✅ 📋 Other
AppPassedFailedSkipped
✅ e2e-local-dev-nest-stable128025
✅ e2e-local-dev-tanstack-start-128025
✅ e2e-local-postgres-nest-stable128025
✅ e2e-local-postgres-tanstack-start-128025
✅ e2e-local-prod-nest-stable128025
✅ e2e-local-prod-tanstack-start-128025
✅ e2e-vercel-prod-tanstack-start126027

📋 View full workflow run

@github-actions

github-actionsBot commented May 6, 2026

Copy link
Copy Markdown
Contributor

📊 Workflow Benchmarks

commit ec311ea · Fri, 10 Jul 2026 01:00:40 GMT · run logs

Backend: vercel · app: nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1333 (+14%)1669 🔴1713 🔴2292 🔴30
TTFShook + stream1567 (+14%)1913 🔴2015 🔴2114 🔴30
STSO1020 steps (1-20)240 (-12%)266 🔴317 🔴400 🔴19
STSO1020 steps (101-120)384 (-8.0%)407 🔴448 🔴499 🔴19
STSO1020 steps (1001-1020)1069 (-0.7%)1095 🔴1220 🔴3609 🔴19
WOstream1333 (+14%)16691713229230
WOhook + stream1567 (+14%)19132015211430
SLstream4386 (-11%)5120 🔴5677 🔴5816 🔴30
SLhook + stream4773 (-5.0%)5360 🔴5768 🔴6025 🔴30
📜 Previous results (3)

82b4b0c

Thu, 09 Jul 2026 23:26:29 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1321 (+19%)1695 🔴1791 🔴1964 🔴30
TTFShook + stream1591 (+24%)2028 🔴2159 🔴2297 🔴30
STSO1020 steps (1-20)271 (±0%)308 🔴371 🔴414 🔴19
STSO1020 steps (101-120)431 (+4.7%)483 🔴624 🔴820 🔴19
STSO1020 steps (1001-1020)843 (-3.5%)882 🔴989 🔴1067 🔴19
WOstream1321 (+19%)16951791196430
WOhook + stream1591 (+24%)20282159229730
SLstream4868 (+4.2%)5638 🔴5679 🔴5942 🔴30
SLhook + stream4625 (-8.0%)5554 🔴5660 🔴5925 🔴30

9a30ea5

Thu, 09 Jul 2026 22:13:10 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream1401 (+40%)1714 🔴1757 🔴1905 🔴30
TTFShook + stream1681 (+34%)1959 🔴2051 🔴2111 🔴30
STSO1020 steps (1-20)287 (+20%)361 🔴463 🔴466 🔴19
STSO1020 steps (101-120)395 (-2.4%)420 🔴444 🔴473 🔴19
STSO1020 steps (1001-1020)890 (-14%)961 🔴1121 🔴1230 🔴19
WOstream1401 (+40%)17141757190530
WOhook + stream1681 (+34%)19592051211130
SLstream4440 (-7.1%)5272 🔴5684 🔴5901 🔴30
SLhook + stream4934 (-1.2%)5587 🔴5671 🔴5792 🔴30

22045b9

Thu, 09 Jul 2026 19:55:20 GMT · run logs

vercel / nextjs-turbopack

MetricScenarioAvg (ms)P75 (ms)P90 (ms)P99 (ms)Samples
TTFSstream584 (-42%)645 🔴702 🔴838 🔴30
TTFShook + stream727 (-42%)761 🔴851 🔴1211 🔴30
STSO1020 steps (1-20)319 (+33%)307 🔴453 🔴1222 🔴19
STSO1020 steps (101-120)611 (+51%)525 🔴635 🔴3406 🔴19
STSO1020 steps (1001-1020)932 (-9.8%)845 🔴1016 🔴3582 🔴19
WOstream584 (-42%)64570283830
WOhook + stream727 (-42%)761851121130
SLstream1384 (-71%)1018 🔴1140 🔴6104 🔴30
SLhook + stream4301 (-14%)5822 🔴5839 🔴6028 🔴30

Avg deltas compare against the most recent benchmark run on main at the time of this run.

Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader)

Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges

🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120

TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request.

Comment threadpackages/core/src/telemetry.ts Outdated
const specifier = ['@opentelemetry', 'api'].join('/');
try {
return await import('@opentelemetry/api');
return (await import(specifier)) as typeof api;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark this as external instead?

@VaguelySeriousVaguelySeriousMay 22, 2026

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review: Note

Correction to my earlier point — we do control bundler config for the framework integrations we ship (packages/sveltekit, packages/next, packages/nitro, packages/nuxt, packages/astro, etc.), and the Next integration already manipulates serverExternalPackages / webpack externals. So we could plausibly add @opentelemetry/api to a per-framework externals list and avoid touching the source.

Why I still think opacifying the specifier is the right call:

  1. Coverage. The source-level fix works for every consumer — including users who don't go through our integrations (direct @workflow/core use, custom Vite/Rollup setups, the workbench builders app's downstream consumers, future frameworks we haven't written an integration for yet). The externalize-per-integration path requires us to maintain a parallel allowlist in 6+ packages and remember to update it whenever the optional-peer surface grows.
  2. Consistency.packages/core/src/runtime/get-world-lazy.ts:54 already uses the runtime-built-specifier trick for the same class of problem (./world.js getting traced into bundles where the sibling doesn't exist). Keeping the two consistent feels right.
  3. Defense in depth. Even if we externalize in our SvelteKit plugin, a user with their own vite.config.ts overriding our config could re-break it. The one-line source fix can't be undone by consumer config.

Happy to additionally externalize in the integrations if you'd prefer belt-and-suspenders, but the source-level fix is what closes the actual reproduction in the PR description.

@karthikscale3karthikscale3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve OTel for the example workflow-server e2e lane

Making the @opentelemetry/api import opaque means bundlers/file tracers no longer include that package. That is intentional for apps without OTel, but it can silently disable SDK/world spans in OTel-enabled apps that only install @vercel/otel. workbench/example is in the Vercel e2e matrix, imports @vercel/otel, but does not directly depend on @opentelemetry/api; from that app, import(["@opentelemetry", "api"].join("/")) still fails with ERR_MODULE_NOT_FOUND even after loading @vercel/otel. Since these helpers catch the failure and turn tracing off, this would break the existing workflow/workflow-server OTel interaction for the example e2e deployment. Please add @opentelemetry/api directly to workbench/example/package.json and ideally document that @vercel/otel alone is not enough for Workflow SDK spans after this change.

…opaque import
The opaque runtime-built specifier fixed the SvelteKit/Rollup build failure but
silently disabled tracing on esbuild self-contained targets (the CLI's
vercel-build-output-api build, Nitro, Astro): those ship a bundle with no
node_modules, so an opaque import can't be inlined and resolves to nothing at
runtime — including the `example` e2e deployment.
Revert core/world-local/world-vercel to a static `import('@opentelemetry/api')`
(esbuild inlines it, nft traces it → tracing works when the peer is present),
and mark `@opentelemetry/api` external in the shared @workflow/rollup transform
plugin so Rollup/Vite framework builds (SvelteKit, Nitro, Nuxt, Astro, Vite)
stay green when the optional peer isn't installed. Verified: the peer-absent
build fails on Vite 6/7 without the external and passes with it.
Also pin @opentelemetry/api directly in workbench/example (aligning it with the
other workbenches) and add a @workflow/rollup regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@VaguelySerious

Copy link
Copy Markdown
MemberAuthor

(AI) Addressed the P1, but changed the approach after digging into the failure mode you flagged.

You were right that the opaque import breaks OTel — and it's actually worse than "only @vercel/otel installed": on the esbuild self-contained targets (the CLI's vercel-build-output-api build that workbench/example uses, plus Nitro/Astro), the .func ships nonode_modules, so an opaque import(specifier) can't be inlined by esbuild and resolves to nothing at runtime — tracing was silently off regardless of whether @opentelemetry/api was in package.json. Adding the dep to the example wouldn't have restored spans there.

So instead of making the import opaque, this now:

  • Reverts the import back to a static import('@opentelemetry/api') in core/world-local/world-vercel, so esbuild inlines it and @vercel/nft traces it → tracing works on every target when the peer is present (verified the example BOA bundle inlines it for the SDK's tracing path).
  • Externalizes@opentelemetry/api in the shared @workflow/rollup plugin so SvelteKit/Nitro/Nuxt/Astro/Vite builds don't fail when the optional peer is absent. Reproduced the original Rollup failed to resolve error on Vite 6/7 and confirmed the external fixes it.

Also did what you asked directly: pinned @opentelemetry/api in workbench/example (it was the outlier — the other workbenches already pin it) and documented the optional peer in the tracing guide.

VaguelySeriousand others added 3 commits July 9, 2026 14:52
Unconditionally externalizing broke the Nitro-based e2e apps (express, fastify,
hono, nitro, tanstack): Nitro's `.output/server` is a self-contained bundle
with no node_modules, so marking the peer external stranded the runtime import
and crashed the server at startup with ERR_MODULE_NOT_FOUND (the `ai` package
imports @opentelemetry/api).
Resolve the specifier first; only fall back to external when it can't be
resolved. Installed → bundles into the self-contained output (Nitro, esbuild)
and traces on node/nft targets; absent → external so Rollup/Vite (SvelteKit)
doesn't fail the build. Verified: express Nitro build now inlines the package
and its server loads with no node_modules present, and the peer-absent Vite 6/7
SSR build stays green via the external fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… absent
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@VaguelySerious@TooTallNate@karthikscale3