Skip to content

feat(cloudflare): Propagate traceparent to RPC calls - via fetch - #19991

Merged
JPeer264 merged 3 commits into
developfrom
jp/rpc-instrument
Apr 10, 2026
Merged

feat(cloudflare): Propagate traceparent to RPC calls - via fetch#19991
JPeer264 merged 3 commits into
developfrom
jp/rpc-instrument

Conversation

@JPeer264

@JPeer264JPeer264 commented Mar 26, 2026

Copy link
Copy Markdown
Member

relates to #19327

related to #16898 (it is not really closing it as we just add context propagation without adding spans for individual calls. It needs to be defined if we need it)

It is important to know that these RPC calls do only work with the .fetch call:

constid=env.MY_DURABLE_OBJECT.idFromName('workflow-test');conststub=env.MY_DURABLE_OBJECT.get(id);awaitstub.fetch(newRequest('http://my-worker/my-do-call'));

This adds RPC fetch calls between:

This works by instrumenting env (via instrumentEnv), which then goes over the bindings and see if there is a DurableObject or a normal Fetcher (full list of current bindings: https://developers.cloudflare.com/workers/runtime-apis/bindings/). This got inspired by how otel-cf-workers instruments their env: https://github.com/evanderkoogh/otel-cf-workers/blob/effeb549f0a4ed1c55ea0c4f0d8e8e37e5494fb3/src/instrumentation/env.ts

With this PR I added a lot of tests to check if trace propagation works (so this PR might look like it added a lot of LoC, but it is mostly tests). So I added it for schedule and queue, but it is not possible for email and tail with wrangler dev.

Potential things to change

Trace propagagtion

I added the addTraceHeaders.ts helper, as there is currently no way to reuse the existing logic (it is baked-in into the fetch instrumentations). It would be nice once #19960 lands that we can reuse it in Cloudflare to reuse existing code. I tried to write couple of tests so we don't have duplicated headers.

Adding extra spans

So there is actually a guide by OTel to add RPC spans, but was talking with someone from the OTel maintainers and they meant that this wouldn't be necessary as we already have an http.server span from out instrumented DurableObjects (and other resources) - so it wouldn't add much of information.

Without RPC span:

Screenshot 2026-03-25 at 10 59 01

With RPC span:

Screenshot 2026-03-25 at 10 55 48

@JPeer264JPeer264 self-assigned this Mar 26, 2026
@linear-code

Copy link
Copy Markdown

@github-actions

github-actionsBot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Semver Impact of This PR

🟡 Minor (new features)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


New Features ✨

Core

  • Add enableTruncation option to Google GenAI integration by andreiborza in #20184
  • Add enableTruncation option to Anthropic AI integration by andreiborza in #20181
  • Add enableTruncation option to LangGraph integration by andreiborza in #20183
  • Add enableTruncation option to LangChain integration by andreiborza in #20182
  • Add enableTruncation option to OpenAI integration by andreiborza in #20167
  • Export a reusable function to add tracing headers by JPeer264 in #20076

Deps

  • Bump hono from 4.12.7 to 4.12.12 by dependabot in #20118
  • Bump defu from 6.1.4 to 6.1.6 by dependabot in #20104

Other

  • (cloudflare) Propagate traceparent to RPC calls - via fetch by JPeer264 in #19991

Bug Fixes 🐛

  • (deno) Avoid inferring invalid span op from Deno tracer by Lms24 in #20128
  • (e2e) Add op check to waitForTransaction in React Router e2e tests by copilot-swe-agent in #20193

Internal Changes 🔧

  • (bugbot) Add rules to flag test-flake-provoking patterns by Lms24 in #20192
  • (deps) Bump axios from 1.13.5 to 1.15.0 in /dev-packages/e2e-tests/test-applications/nestjs-basic by dependabot in #20179
  • (size-limit) Bump failing size limit scenario by Lms24 in #20186

🤖 This preview updates automatically when you update the PR.

@github-actions

github-actionsBot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

size-limit report 📦

PathSize% ChangeChange
@sentry/browser25.72 kB--
@sentry/browser - with treeshaking flags24.21 kB--
@sentry/browser (incl. Tracing)42.73 kB--
@sentry/browser (incl. Tracing, Profiling)47.35 kB--
@sentry/browser (incl. Tracing, Replay)81.54 kB--
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags71.11 kB--
@sentry/browser (incl. Tracing, Replay with Canvas)86.25 kB--
@sentry/browser (incl. Tracing, Replay, Feedback)98.45 kB--
@sentry/browser (incl. Feedback)42.51 kB--
@sentry/browser (incl. sendFeedback)30.39 kB--
@sentry/browser (incl. FeedbackAsync)35.38 kB--
@sentry/browser (incl. Metrics)27.04 kB--
@sentry/browser (incl. Logs)27.18 kB--
@sentry/browser (incl. Metrics & Logs)27.86 kB--
@sentry/react27.48 kB--
@sentry/react (incl. Tracing)45.05 kB--
@sentry/vue30.56 kB--
@sentry/vue (incl. Tracing)44.59 kB--
@sentry/svelte25.74 kB--
CDN Bundle28.41 kB--
CDN Bundle (incl. Tracing)43.75 kB--
CDN Bundle (incl. Logs, Metrics)29.78 kB--
CDN Bundle (incl. Tracing, Logs, Metrics)44.83 kB--
CDN Bundle (incl. Replay, Logs, Metrics)68.59 kB--
CDN Bundle (incl. Tracing, Replay)80.64 kB--
CDN Bundle (incl. Tracing, Replay, Logs, Metrics)81.66 kB--
CDN Bundle (incl. Tracing, Replay, Feedback)86.17 kB--
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics)87.2 kB--
CDN Bundle - uncompressed82.99 kB--
CDN Bundle (incl. Tracing) - uncompressed129.77 kB--
CDN Bundle (incl. Logs, Metrics) - uncompressed87.14 kB--
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed133.19 kB--
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed210.12 kB--
CDN Bundle (incl. Tracing, Replay) - uncompressed246.65 kB--
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed250.05 kB--
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed259.56 kB--
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed262.95 kB--
@sentry/nextjs (client)47.47 kB--
@sentry/sveltekit (client)43.2 kB--
@sentry/node-core57.86 kB+0.02%+7 B 🔺
@sentry/node174.86 kB+0.01%+10 B 🔺
@sentry/node - without tracing97.97 kB+0.03%+22 B 🔺
@sentry/aws-serverless115.22 kB+0.02%+19 B 🔺

View base workflow run

@timfishtimfish left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@JPeer264JPeer264 changed the title feat(cloudflare): Propagate traceparent to RPC callsfeat(cloudflare): Propagate traceparent to RPC calls - via fetchMar 31, 2026
@Lms24

Copy link
Copy Markdown
Member

It would be nice once #19960 lands that we can reuse it in Cloudflare to reuse existing code. I tried to write couple of tests so we don't have duplicated headers.

FYI, #19960 landed and I agree! Right now, our various request instrumentations combine the "logic"/"algorithm" code with platform specific code how to set headers. I think it would be good if we could strip out the algorithm part into a common helper, and only leave the platform-specific parts to the individual instrumentations. From my PoV a good improvement but something we can also tackle at a later point. Happy to leave this up to you :)

Comment threadpackages/cloudflare/src/utils/addTraceHeaders.ts Outdated
Comment threadpackages/cloudflare/src/utils/addTraceHeaders.ts Outdated
Comment on lines +13 to +15
const newInit = addTraceHeaders(input, init);

return fetchFn(input, newInit);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m/q: is this how we add our tracing headers in our core/browser fetch instrumentation as well? Just wanted to double check because injecting headers into fetch arguments is a bit tricky with the various forms of input and init and the precedence.

I think adding headers to init is fine. Though us deep-copying headers from input to init, and then init again might be unexpected if one of the objects is reused by users across multiple requests. Anyway, not saying something is off but that we should double check that this is how we do it in the other SDKs as well.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, that would be indeed a bad behavior - I'll change the behavior to match the others

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Lms24 the headers are now attached with the same function as the other SDKs: _INTERNAL_getTracingHeadersForFetchRequest.

Comment threadpackages/cloudflare/src/utils/addTraceHeaders.ts Outdated
Comment threadpackages/cloudflare/src/utils/addTraceHeaders.ts Outdated
@JPeer264

JPeer264 commented Apr 1, 2026

Copy link
Copy Markdown
MemberAuthor

@Lms24 I just reworked the way how tracing headers are generated. I actually exported getTracingHeadersForFetchRequest from core, which is used inside instrumentFetchRequests in core. I actually saw that mergeBaggageHeaders is not applied there, but I think this should be added (maybe in a follow up PR?!).

I can also outsource getTracingHeadersForFetchRequest into its own PR if wanted, to make this PR a little bit more focused on one thing. (Edit: I actually outsourced this: #20076)

@JPeer264
JPeer264 requested a review from Lms24April 1, 2026 15:19

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@github-actions

github-actionsBot commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

node-overhead report 🧳

Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.

ScenarioRequests/s% of BaselinePrev. Requests/sChange %
GET Baseline9,137-8,855+3%
GET With Sentry1,70619%1,740-2%
GET With Sentry (error only)5,93965%6,020-1%
POST Baseline1,190-1,163+2%
POST With Sentry59550%588+1%
POST With Sentry (error only)1,03987%1,035+0%
MYSQL Baseline3,185-3,155+1%
MYSQL With Sentry39812%440-10%
MYSQL With Sentry (error only)2,61182%2,567+2%

View base workflow run

JPeer264 added a commit that referenced this pull request Apr 7, 2026
part of #15942 This is a basic instrumentation for `WorkerEntrypoint`. There is no
extra `instrumentWorkerEntrypoint` exported, and should work directly
with `withSentry` OOTB:
```js
class MyWorker extends WorkerEntrypoint {}
export default Sentry.withSentry(() => ({}), MyWorker)
```
To support the full `WorkerEntrypoint` there is the need to instrument
[RPC
methods](#16898).
Without that we can't fully support `WorkerEntrypoint`s but just the
basics of it. This can be added once #19991 lands (it is safe to review
now and instrument the env in a separate PR)
--- If you look at the instrumentations themselves, there is no
`isInstrumented` or `markAsInstrumented`. The reason is that on the one
hand this is not required, as every request is spawning a new instance
of the `WorkerEntrypoint` and on the other hand it wouldn't work as it
would mark it as instrumented, but on the request the `fetch` is
overridden and not instrumented.
JPeer264 added a commit that referenced this pull request Apr 9, 2026
This PR is an extraction of #19991 It basically exports `getTracingHeadersForFetchRequest`, which was
previously only exported for testing, but offers a great functionality
if you want to add tracing headers to a request. I renamed it as
`addTracingHeadersToFetchRequest` sounded a little misleading, as it
didn't really add headers to the request, as it returned the extracted
headers from the request (or init, if there are any).
### Open question
I added `@hidden` and `@internal` to it, not sure if this is an approach
we follow. I'm ok to remove it from the jsdoc
@JPeer264
JPeer264force-pushed the jp/rpc-instrument branch 2 times, most recently from bc033a9 to f62418dCompareApril 9, 2026 19:03
@JPeer264
JPeer264 merged commit a641812 into developApr 10, 2026
63 checks passed
@JPeer264
JPeer264 deleted the jp/rpc-instrument branch April 10, 2026 13:51
JPeer264 added a commit that referenced this pull request Apr 16, 2026
…pagation (#20345)
follow up to #19991
It is better to release it first with an option to be enabled, that
would then also be in line with #20343, otherwise `.fetch()` RPC calls
would work without any option and the actual Cap'n'Proto RPC calls
wouldn't work without. That would be an odd experience.
### New option: `enableRpcTracePropagation`
> `instrumentPrototypeMethods` has been deprecated in favor of
`enableRpcTracePropagation`
Replaces the deprecated `instrumentPrototypeMethods` option with a
clearer name that describes what it actually does. This option must be
enabled on **both** the caller (Worker) and receiver (Durable Object)
sides for trace propagation to work.
It is also worth to mention that the implementation of "instrumenting
prototype methods" has changed to a Proxy.
```ts
// Worker side
export default Sentry.withSentry(
(env) => ({
dsn: env.SENTRY_DSN,
enableRpcTracePropagation: true,
}),
handler,
);
// Durable Object side
export const MyDurableObject = Sentry.instrumentDurableObjectWithSentry(
(env) => ({
dsn: env.SENTRY_DSN,
enableRpcTracePropagation: true,
}),
MyDurableObjectBase,
);
```
andreiborza pushed a commit that referenced this pull request Apr 20, 2026
…pagation (#20345)
follow up to #19991
It is better to release it first with an option to be enabled, that
would then also be in line with #20343, otherwise `.fetch()` RPC calls
would work without any option and the actual Cap'n'Proto RPC calls
wouldn't work without. That would be an odd experience.
### New option: `enableRpcTracePropagation`
> `instrumentPrototypeMethods` has been deprecated in favor of
`enableRpcTracePropagation`
Replaces the deprecated `instrumentPrototypeMethods` option with a
clearer name that describes what it actually does. This option must be
enabled on **both** the caller (Worker) and receiver (Durable Object)
sides for trace propagation to work.
It is also worth to mention that the implementation of "instrumenting
prototype methods" has changed to a Proxy.
```ts
// Worker side
export default Sentry.withSentry(
(env) => ({
dsn: env.SENTRY_DSN,
enableRpcTracePropagation: true,
}),
handler,
);
// Durable Object side
export const MyDurableObject = Sentry.instrumentDurableObjectWithSentry(
(env) => ({
dsn: env.SENTRY_DSN,
enableRpcTracePropagation: true,
}),
MyDurableObjectBase,
);
```
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@JPeer264@Lms24@timfish