Uh oh!
There was an error while loading. Please reload this page.
fix(typescript): enable ADOT instrumentation for TypeScript agents - #1893
Open
jariy17 wants to merge 1 commit into
Open
fix(typescript): enable ADOT instrumentation for TypeScript agents#1893jariy17 wants to merge 1 commit into
jariy17 wants to merge 1 commit into
Conversation
TypeScript agents received no OpenTelemetry instrumentation, so spans from any third-party instrumentation library were silently dropped and nothing appeared in CloudWatch. Three gaps, each sufficient on its own: - `enableOtel` was hardcoded off for TypeScript in the render config - the TypeScript Dockerfile had no instrumentation branch at all, so setting `instrumentation.enableOtel` on a TS agent was a silent no-op - neither TS template depended on an OTel SDK — only the no-op `@opentelemetry/api`, which discards spans unless a provider is registered This is a regression: 580cd10 ("remove OTEL, session storage, and gateway from TS templates", #981) deleted the previous working implementation. Node has no `opentelemetry-instrument` equivalent, so the Dockerfile preloads the ADOT distro via NODE_OPTIONS instead of wrapping the entrypoint. The CodeZip path is handled in @aws/agentcore-cdk. Also force-enables the Vercel AI SDK's own telemetry. ADOT's Vercel instrumentation patches the `ai` module at import time, which never fires under CodeZip because esbuild inlines `ai` into the bundle, leaving no import to intercept. Toggling the SDK's telemetry works regardless of bundling: ai@6 pulls the global tracer that ADOT registers. Without this, Vercel agents produced only network-level spans — no model spans, no gen_ai.* attributes. Strands is unaffected because its template self-instruments via @opentelemetry/api. Extends the Dockerfile render test to cover TypeScript; it previously only exercised the Python Dockerfile, which is why this regressed unnoticed. Verified on a live account: all four agent shapes (Strands/VercelAI x CodeZip/Container) produce X-Ray spans, and both Vercel shapes now emit `chat <model>` spans carrying gen_ai.request.model, gen_ai.response.model, and gen_ai.response.finish_reasons.
Contributor
Package TarballHow to installgh release download pr-1893-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.25.0.tgz |
Contributor
Claude Security Review: no high-confidence findings. (run) |
Contributor
Coverage Report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#1892
Depends on aws/agentcore-l3-cdk-constructs#311 (CodeZip path)
Problem
agentcore deploysets up no ADOT/OpenTelemetry instrumentation for TypeScript agents. Spans from any third-party instrumentation library are silently dropped and nothing appears in CloudWatch. Three independent gaps, each sufficient on its own:enableOtelhardcoded off for TypeScript —schema-mapper.ts:{{#if enableOtel}}; the TS one ended unconditionally atCMD ["npx", "tsx", "main.ts"]. So settinginstrumentation.enableOtel: trueon a TS agent was a silent no-op — there was nothing for it to render — while the schema documents it as wrapping the entrypoint.@opentelemetry/api(the no-op API surface, which discards spans unless a provider is registered); Vercel AI carried nothing.This is a regression
580cd10 — "fix(templates): remove OTEL, session storage, and gateway from TS templates", merged in #981 — deleted the working implementation: both
otel-register.tsfiles, their imports frommain.ts, seven@opentelemetry/*dependencies, and it flippedschema-mapper.tsto exclude TypeScript. The OTel removal was one commit inside a broader TS template PR, so it isn't visible from the PR title.Changes
schema-mapper.ts— stop excluding TypeScript fromenableOtelcontainer/typescript/Dockerfile— add the{{#if enableOtel}}branch. Node has noopentelemetry-instrumentequivalent, so it preloads the ADOT distro viaNODE_OPTIONSrather than wrapping the entrypoint@aws/aws-distro-opentelemetry-node-autoinstrumentationvercelai/base/main.ts— force-enable the Vercel AI SDK's own telemetry (see below)dockerfile-render.test.ts— extend to cover TypeScriptThe CodeZip path needs a different mechanism (no Dockerfile to hook) and lives in aws/agentcore-l3-cdk-constructs#311.
Why the Vercel template needs an explicit telemetry toggle
Turning on ADOT was necessary but not sufficient for Vercel AI. With instrumentation loaded and traces flowing, Vercel agents still produced only network-level spans — no model spans, no
gen_ai.*attributes — while Strands produced a full GenAI tree from the same runtime.Two compounding causes:
aimodule at import time. Both templates are"type": "module"and load instrumentation via--require(a CJS hook), which never observes an ESMimport ... from 'ai'.aiinto the bundle, so there is no module resolution event left to intercept. An ESM loader hook would have fixed Container and silently missed CodeZip.Setting
experimental_telemetryon the call site sidesteps both.ai@6resolves the global tracer (trace.getTracer('ai')from@opentelemetry/api, shared viaglobalThis), which ADOT registers at startup — so it works regardless of bundling or module system. Gated onAGENT_OBSERVABILITY_ENABLED === 'true', matching ADOT's own gate.Strands needs no equivalent: its template self-instruments via
@opentelemetry/api.Verification
Deployed all four TypeScript agent shapes to a live account, invoked each 5 times, waited for ingestion, and queried X-Ray.
All four log
AWS Distro of OpenTelemetry automatic instrumentation started successfullyand produce X-Ray spans including downstream Bedrock calls.Vercel AI, before vs after:
Attributes now present:
gen_ai.request.model,gen_ai.response.model,gen_ai.response.finish_reasons,gen_ai.operation.name,gen_ai.provider.name,gen_ai.system. Confirmed on both CodeZip and Container.Tests: asset suite green (141), including the new TS Dockerfile render cases.
Known cosmetic issue, not addressed here
Runtime logs show
@aws/...-instrumentation-vercel-ai Failed to register VercelAISpanProcessor. It is harmless and unrelated: ADOT'ssetTracerProvideris called once before the real provider exists (logged atwarn) and again after, when it succeeds (logged atdebug, so invisible in production). It also appears for Strands agents, which have noaipackage installed at all. Worth an upstream log-level fix onaws-observability/aws-otel-js-instrumentation; nothing is lost.Follow-up
If LangChain or OpenAI-Agents TypeScript templates are added later, they will hit the same ESM/bundling wall — their ADOT instrumentation patches modules identically and will be equally inert. Each will need a per-SDK telemetry toggle, or an
--import-based ESM loader hook on the Container path.