bug(typescript): agentcore deploy sets up no ADOT/OTel instrumentation for TypeScript agents, silently dropping all 3p telemetry

[jariy17]

Description

agentcore deploy does not set up any OpenTelemetry / ADOT instrumentation for TypeScript agents. Python agents get zero-code instrumentation via opentelemetry-instrument; TypeScript agents get nothing at any layer. As a result, spans emitted by any 3p instrumentation library (Strands, Vercel AI SDK, or a user's own @opentelemetry/api calls) are silently dropped, and nothing appears in CloudWatch / X-Ray Transaction Search.

This is a regression, not a never-implemented feature — the wiring existed and was removed (see below).

Steps to Reproduce

1. agentcore create --language TypeScript --sdk Strands --protocol HTTP --build Container
2. agentcore deploy
3. Invoke the deployed agent so it produces LLM/tool calls
4. Open CloudWatch → Application Signals / X-Ray Transaction Search and look for traces from the agent

Also observable statically, without deploying:

• Generated Dockerfile ends in CMD ["npx", "tsx", "main.ts"] — no instrumentation wrapper
• Generated package.json contains no OTel SDK, only the no-op @opentelemetry/api

Expected Behavior

TypeScript agents get zero-code ADOT instrumentation at parity with Python, so that 3p instrumentation libraries emitting via @opentelemetry/api export to the collector and show up in CloudWatch without any application code changes.

Actual Behavior

No instrumentation is configured. Three independent gaps, each sufficient on its own to produce zero telemetry:

1. enableOtel is hardcoded off for TypeScript — src/cli/operations/agent/generate/schema-mapper.ts:284

const enableOtel = !isMcp && config.language !== 'TypeScript';

2. The TypeScript Dockerfile has no instrumentation branch at all — src/assets/container/typescript/Dockerfile:25

CMD ["npx", "tsx", "main.ts"]

Compare src/assets/container/python/Dockerfile:40-44, which does branch:

{{#if enableOtel}}
CMD ["opentelemetry-instrument", "python", "-m", "{{entrypoint}}"]
{{else}}
CMD ["python", "-m", "{{entrypoint}}"]
{{/if}}

Because no {{#if enableOtel}} block exists in the TS Dockerfile, setting instrumentation.enableOtel: true on a TypeScript agent has no effect — there is nothing for it to render. The schema at src/schema/schemas/agent-env.ts:194 defaults it to true and documents it as "the runtime entrypoint is wrapped with opentelemetry-instrument", which is silently untrue for TypeScript.

3. No OTel SDK dependency or bootstrap in the TS templates. src/assets/typescript/http/strands/base/package.json carries only @opentelemetry/api — the no-op API surface, which discards all spans unless a global provider is registered. src/assets/typescript/http/vercelai/base/package.json has no OTel dependency at all. Neither template registers a provider.

The CodeZip path is affected too: TS CodeZip builds are bundled with esbuild (src/lib/packaging/node.ts:95) and have no Dockerfile to hook, so they need the instrumentation applied via runtime env vars rather than a CMD change.

Origin of the regression

Commit 580cd10b — "fix(templates): remove OTEL, session storage, and gateway from TS templates", merged in #981 (2026-05-14) — removed the working implementation:

• deleted src/assets/typescript/http/strands/base/otel-register.ts and the vercelai equivalent, which registered a NodeTracerProvider + MeterProvider with OTLP exporters
• removed import './otel-register.js'; from both main.ts files
• removed 7 @opentelemetry/* dependencies from both package.json files
• flipped schema-mapper.ts to exclude TypeScript

The OTel removal was one commit inside a broader TS template/streaming PR, so it isn't visible from the PR title.

Interaction with #1270 (important)

Fixing the CLI alone is not sufficient to get traces into CloudWatch. Per #1270, AgentCore Runtime starts the ADOT collector sidecar on localhost:4318 only when it detects opentelemetry-instrument in the container entrypoint, which is Python-specific. A TypeScript entrypoint never matches, so nothing listens on :4318 and OTLP export fails silently even with a correctly registered SDK.

These two need to land together for end-to-end telemetry. Adopting a --require-based ADOT entrypoint (below) may also give the Runtime a detectable signal for starting the sidecar, which is worth confirming with the Runtime team.

Suggested fix

ADOT for Node exists and is current: @aws/aws-distro-opentelemetry-node-autoinstrumentation@0.12.0. True zero-code parity with Python is a NODE_OPTIONS=--require @aws/aws-distro-opentelemetry-node-autoinstrumentation/register entrypoint — a single dependency and no hand-maintained otel-register.ts, unlike the reverted implementation.

Touch points:

• src/cli/operations/agent/generate/schema-mapper.ts:284 — stop excluding TypeScript
• src/assets/container/typescript/Dockerfile — add the {{#if enableOtel}} branch
• both TS template package.json files — add the ADOT distro dependency
• TS CodeZip path — set the instrumentation env vars on the runtime (no Dockerfile to hook)
• src/assets/__tests__/dockerfile-render.test.ts — currently only exercises the Python Dockerfile; add TS coverage so this cannot silently regress again

CLI Version

0.25.0 (repo package.json, at commit 8fba816)

[jariy17]

Correction and resolution

Fixed in #1893 (CLI) + aws/agentcore-l3-cdk-constructs#311 (CodeZip path). Verified end to end on a live account.

Two claims in the original report were wrong, and one gap was missing. Recording them here since they changed the fix.

1. The CodeZip mechanism was described wrong

The report says CodeZip "needs the instrumentation applied via runtime env vars." Env vars are necessary but not sufficient. The real blocker was a fourth gap, in a different repo: AgentCoreRuntime excluded Node explicitly —

const entryPoint = enableOtel && !isNode ? ['opentelemetry-instrument', entrypointPath] : [entrypointPath];

— so no CLI-only change could fix CodeZip.

Setting NODE_OPTIONS alone also doesn't work. Two constraints, both found by testing:

• The zip writer skips any directory named node_modules at any depth (EXCLUDED_ENTRIES), so the distro can't be staged as a normal dependency tree. This is what the existing _deps directory already exists to work around.
• --require runs before the esbuild banner installs its _deps resolver, so a bare specifier doesn't resolve during preload.

The fix stages the distro's transitive closure flat under _deps and emits a resolver shim that NODE_OPTIONS preloads.

2. #1270 is not a blocker

The report claims the CLI fix is insufficient without #1270, on the theory that the runtime starts its OTLP collector only when it detects opentelemetry-instrument in the entrypoint. That prediction was wrong. Traces from NODE_OPTIONS-based Node agents reach X-Ray fine — confirmed for all four TypeScript agent shapes. #1270 may still be worth doing for .NET/Java, but it does not gate TypeScript.

3. Missing gap: Vercel AI emitted no model spans

Not in the original report. With ADOT loaded and traces flowing, Vercel AI agents still produced only network-level spans — no model spans, no gen_ai.* attributes — while Strands produced a full GenAI tree from the same runtime.

Cause: ADOT's Vercel instrumentation patches the ai module at import time. Both templates are "type": "module" and load instrumentation via --require (a CJS hook), which never sees an ESM import ... from 'ai'. Under CodeZip it can't work at all — esbuild inlines ai into the bundle, so there's no module resolution event to intercept. (An ESM loader hook would have fixed Container and silently missed CodeZip.)

Fix: set experimental_telemetry on the call site. ai@6 resolves the global tracer that ADOT registers, so it works regardless of bundling or module system. Strands needs no equivalent — its template self-instruments via @opentelemetry/api.

Before / after:

before: SdkVercelZip.DEFAULT
 - bedrock-runtime.us-east-1.amazonaws.com:443

after: SdkVercelZip.DEFAULT
 - chat us.anthropic.claude-sonnet-4-5-20250929-v1:0
 - chat us.anthropic.claude-sonnet-4-5-...
 - bedrock-runtime.us-east-1.amazonaws.com:443

What was right

The three CLI gaps as described, and the regression origin (580cd10 in #981). Also worth restating: instrumentation.enableOtel was a silent no-op for TypeScript — the schema documents it as wrapping the entrypoint, but the TS Dockerfile had no {{#if enableOtel}} block for it to render.

Unrelated cosmetic warning

Logs show @aws/...-instrumentation-vercel-ai Failed to register VercelAISpanProcessor. Harmless: ADOT's setTracerProvider is called once before the real provider exists (logged at warn) and again after, when it succeeds (logged at debug, invisible in production). It appears for Strands agents too, which have no ai package installed. Upstream log-level bug on aws-observability/aws-otel-js-instrumentation; nothing is lost.

Verification

Four agent shapes (Strands / Vercel AI x CodeZip / Container) deployed, invoked 5x each, X-Ray queried after ingestion. All log AWS Distro of OpenTelemetry automatic instrumentation started successfully and produce spans including downstream Bedrock calls. Both Vercel shapes now emit chat <model> spans with gen_ai.request.model, gen_ai.response.model, and gen_ai.response.finish_reasons.