feat(ai): emit cost + full usage on otel spans by season179 · Pull Request #747 · TanStack/ai

season179 · 2026-06-11T15:31:17Z

Summary

Fixes#721.

otelMiddleware only emitted gen_ai.usage.input_tokens / gen_ai.usage.output_tokens, even though TokenUsage already carries provider-reported cost, total tokens, cache/reasoning breakdowns, and duration-based billing (the cost fields landed in #654). Backends like PostHog had to re-derive cost from tokens × their own price table, losing cache discounts and gateway markup (OpenRouter), and duration-billed activities had no cost signal at all.

What changed

A shared usageAttributes() helper now builds the full attribute set at all three emission sites (RUN_FINISHED chunk, onUsage, onFinish rollup). Every field is guarded, so spans are unchanged when a provider doesn't report it:

Attribute	Source	Convention
`gen_ai.usage.total_tokens`	`totalTokens`	de-facto (PostHog/LiteLLM consume directly)
`gen_ai.usage.cost`	`cost`	de-facto (PostHog consumes directly)
`gen_ai.usage.cache_read.input_tokens`	`promptTokensDetails.cachedTokens`	official GenAI semconv
`gen_ai.usage.cache_creation.input_tokens`	`promptTokensDetails.cacheWriteTokens`	official GenAI semconv
`gen_ai.usage.reasoning.output_tokens`	`completionTokensDetails.reasoningTokens`	official GenAI semconv
`tanstack.ai.usage.duration_seconds`	`durationSeconds`	TanStack-namespaced (no semconv)
`tanstack.ai.usage.upstream_cost` / `_input_cost` / `_output_cost`	`costDetails`	TanStack-namespaced (no semconv)

Deliberately out of scope: unitsBilled and per-modality token breakdowns (media activities don't flow through chat middleware — that's #720), and providerUsageDetails (provider-shaped bag, unsafe to spread onto spans).

Tests

Unit: 5 new cases in packages/ai/tests/middlewares/otel.test.ts covering all three emission sites, absent-field omission, and empty detail objects.
E2E: new /api/otel-usage route drives the existing openai-usage-details (cache/reasoning/totals) and openrouter-cost (cost/cost split) aimock mounts through otelMiddleware with an in-memory capture tracer; middleware.spec.ts asserts the attributes land on both iteration and root spans.
Docs: attribute table + naming-convention note in docs/advanced/otel.md.

pnpm test:pr and the full middleware E2E suite pass locally.

Summary by CodeRabbit

New Features
- OpenTelemetry spans now emit comprehensive GenAI usage attributes when providers supply them, including total tokens, cost (with upstream/gateway splits), cache/read/write and reasoning token breakdowns, and duration-based billing.
Documentation
- Expanded OpenTelemetry documentation with the updated usage attribute reference, naming conventions, and notes on conditional emission.
Tests / E2E
- Added E2E endpoint and tests to validate full usage mapping, correct aggregation from iteration to root spans, and preservation of zero-valued fields.

coderabbitai · 2026-06-11T15:31:36Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fffd042f-2f86-44c0-8627-d0da11e244cd

📥 Commits

Reviewing files that changed from the base of the PR and between c7df2a3 and 8ea04a2.

📒 Files selected for processing (1)

packages/ai/tests/middlewares/otel.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

packages/ai/tests/middlewares/otel.test.ts

📝 Walkthrough

Walkthrough

otelMiddleware now emits complete provider-reported token usage, cost, cache/reasoning token breakdowns, and duration-based billing attributes to OpenTelemetry spans. A new usageAttributes() helper centralizes attribute construction and is called at three span lifecycle points. Unit tests and E2E validation cover OpenAI and OpenRouter semantics.

Changes

OpenTelemetry full usage span attributes

Layer / File(s)	Summary
Full usage attributes helper `packages/ai/src/middlewares/otel.ts`	New `usageAttributes(TokenUsage)` helper builds complete `gen_ai.usage.*` span attributes including optional cost, total tokens, cache/reasoning breakdowns, duration, and upstream cost fields. Integrated at three points: `onChunk` (RUN_FINISHED), `onUsage`, and `onFinish`.
Unit test coverage for full usage `packages/ai/tests/middlewares/otel.test.ts`	New test suite with `fullUsage` fixture and `expectFullUsageAttrs` assertion helper. Tests emit full attributes from both `onChunk` and `onUsage`, validate root span aggregation on `onFinish`, and assert optional attributes are omitted when providers do not report them.
E2E API endpoint and tracer capture `testing/e2e/src/routes/api.otel-usage.ts`, `testing/e2e/src/routeTree.gen.ts`	New POST `/api/otel-usage` endpoint with provider selection (OpenAI/OpenRouter). Custom in-memory span capture tracer records start/end/attribute events for assertion. Route tree generated with full registration across type interfaces and module augmentations.
End-to-end span emission tests `testing/e2e/tests/middleware.spec.ts`	E2E tests validate OpenAI (total, cached, reasoning tokens) and OpenRouter (cost, upstream cost breakdown) span attributes on iteration and root spans.
Documentation and metadata `.changeset/otel-full-usage-emission.md`, `docs/advanced/otel.md`, `docs/config.json`	Changeset, expanded attribute reference table, conditional emission clarification, and config timestamp update.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

TanStack/ai#654: Establishes provider-reported usage.cost and usage.costDetails normalization that this PR emits to spans via the new usageAttributes helper.

Suggested reviewers

tombeckenham

Poem

🐰 Spans now bloom with tokens and cost,
Cache discounts no longer lost,
From OpenAI dreams to Router streams—
Each usage detail flows like moonbeams!
The tracer captures, tests will dance,
Full usage given its rightful chance. 🌙

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and accurately summarizes the main change: emitting cost and full TokenUsage details on OpenTelemetry spans via otelMiddleware.
Description check	✅ Passed	The description comprehensively covers motivation, implementation details, scope boundaries, and testing, with reference to the Contributing guide checklist structure provided by the template.
Linked Issues check	✅ Passed	All objectives from issue `#721` are addressed: cost and total tokens are emitted, cache/reasoning details via GenAI semconv attributes, duration fields, cost breakdowns, and all fields are properly guarded.
Out of Scope Changes check	✅ Passed	All changes directly support the linked issue objectives. Test infrastructure updates (routeTree.gen.ts, E2E routes/tests) and documentation are appropriate for validating the new functionality.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

…g details) otelMiddleware only emitted gen_ai.usage.input_tokens/output_tokens even though TokenUsage already carries provider-reported cost, total tokens, cache/reasoning breakdowns, and duration-based billing. Backends like PostHog had to re-derive cost from their own price tables, losing cache discounts and gateway markup (OpenRouter), and duration-billed activities had no cost signal at all. A shared usageAttributes() helper now builds the full guarded attribute set at all three emission sites (RUN_FINISHED chunk, onUsage, onFinish rollup): - gen_ai.usage.total_tokens / gen_ai.usage.cost (de-facto extensions consumed directly by PostHog and LiteLLM-style backends) - gen_ai.usage.cache_read.input_tokens, cache_creation.input_tokens, reasoning.output_tokens (official GenAI semconv names) - tanstack.ai.usage.duration_seconds and the upstream cost split (no semconv equivalent exists) E2E: new /api/otel-usage route drives the existing openai-usage-details and openrouter-cost aimock mounts through otelMiddleware with a local capture tracer; middleware.spec.ts asserts the attributes land on iteration and root spans. FixesTanStack#721

nx-cloud · 2026-06-16T09:03:03Z

View your CI Pipeline Execution ↗ for commit 8ea04a2

Command	Status	Duration	Result
`nx run-many --targets=build --exclude=examples/...`	✅ Succeeded	1m 13s	View ↗

☁️ Nx Cloud last updated this comment at 2026-06-16 09:05:27 UTC

pkg-pr-new · 2026-06-16T09:05:39Z

Open in StackBlitz

@tanstack/ai

npm i https://pkg.pr.new/@tanstack/ai@747

@tanstack/ai-angular

npm i https://pkg.pr.new/@tanstack/ai-angular@747

@tanstack/ai-anthropic

npm i https://pkg.pr.new/@tanstack/ai-anthropic@747

@tanstack/ai-client

npm i https://pkg.pr.new/@tanstack/ai-client@747

@tanstack/ai-code-mode

npm i https://pkg.pr.new/@tanstack/ai-code-mode@747

@tanstack/ai-code-mode-skills

npm i https://pkg.pr.new/@tanstack/ai-code-mode-skills@747

@tanstack/ai-devtools-core

npm i https://pkg.pr.new/@tanstack/ai-devtools-core@747

@tanstack/ai-elevenlabs

npm i https://pkg.pr.new/@tanstack/ai-elevenlabs@747

@tanstack/ai-event-client

npm i https://pkg.pr.new/@tanstack/ai-event-client@747

@tanstack/ai-fal

npm i https://pkg.pr.new/@tanstack/ai-fal@747

@tanstack/ai-gemini

npm i https://pkg.pr.new/@tanstack/ai-gemini@747

@tanstack/ai-grok

npm i https://pkg.pr.new/@tanstack/ai-grok@747

@tanstack/ai-groq

npm i https://pkg.pr.new/@tanstack/ai-groq@747

@tanstack/ai-isolate-cloudflare

npm i https://pkg.pr.new/@tanstack/ai-isolate-cloudflare@747

@tanstack/ai-isolate-node

npm i https://pkg.pr.new/@tanstack/ai-isolate-node@747

@tanstack/ai-isolate-quickjs

npm i https://pkg.pr.new/@tanstack/ai-isolate-quickjs@747

@tanstack/ai-mcp

npm i https://pkg.pr.new/@tanstack/ai-mcp@747

@tanstack/ai-ollama

npm i https://pkg.pr.new/@tanstack/ai-ollama@747

@tanstack/ai-openai

npm i https://pkg.pr.new/@tanstack/ai-openai@747

@tanstack/ai-openrouter

npm i https://pkg.pr.new/@tanstack/ai-openrouter@747

@tanstack/ai-preact

npm i https://pkg.pr.new/@tanstack/ai-preact@747

@tanstack/ai-react

npm i https://pkg.pr.new/@tanstack/ai-react@747

@tanstack/ai-react-ui

npm i https://pkg.pr.new/@tanstack/ai-react-ui@747

@tanstack/ai-solid

npm i https://pkg.pr.new/@tanstack/ai-solid@747

@tanstack/ai-solid-ui

npm i https://pkg.pr.new/@tanstack/ai-solid-ui@747

@tanstack/ai-svelte

npm i https://pkg.pr.new/@tanstack/ai-svelte@747

@tanstack/ai-utils

npm i https://pkg.pr.new/@tanstack/ai-utils@747

@tanstack/ai-vue

npm i https://pkg.pr.new/@tanstack/ai-vue@747

@tanstack/ai-vue-ui

npm i https://pkg.pr.new/@tanstack/ai-vue-ui@747

@tanstack/openai-base

npm i https://pkg.pr.new/@tanstack/openai-base@747

@tanstack/preact-ai-devtools

npm i https://pkg.pr.new/@tanstack/preact-ai-devtools@747

@tanstack/react-ai-devtools

npm i https://pkg.pr.new/@tanstack/react-ai-devtools@747

@tanstack/solid-ai-devtools

npm i https://pkg.pr.new/@tanstack/solid-ai-devtools@747

commit: 8ea04a2

Resolves overlap from the now-merged TanStack#747 (kept the branch's extraction of usageAttributes/firstNumber/errorMessage into shared modules) and TanStack#763's middleware capability primitives (kept both the observability and capability exports in the package barrel). routeTree.gen.ts unioned; will be regenerated.

season179 marked this pull request as draft June 11, 2026 15:33

season179 force-pushed the feat/otel-full-usage-emission branch from 10d8c42 to c7df2a3Compare June 11, 2026 15:45

season179 marked this pull request as ready for review June 11, 2026 15:47

This was referenced Jun 15, 2026
feat(ai): activity-agnostic observability hook for media activities (#720) season179/ai#1
Closed
feat(ai): generation middleware for media observability (#720) #760
Merged

AlemTuzlak approved these changes Jun 16, 2026
View reviewed changes

ci: apply automated fixes
8ea04a2

AlemTuzlak merged commit 7103348 into TanStack:mainJun 16, 2026
10 checks passed

github-actionsBot mentioned this pull request Jun 16, 2026
ci: Version Packages #770
Merged

season179 deleted the feat/otel-full-usage-emission branch June 16, 2026 14:48

This was referenced Jun 18, 2026
feat: cost for OpenRouter image models #782
Merged
fix(ai): forward token usage on the structured-output fallback path #789
Merged

coderabbitaiBot mentioned this pull request Jul 4, 2026
feat: add self-describing billed usage to TokenUsage #896
Merged

This was referenced Jul 12, 2026
fix(ai): roll up OpenTelemetry usage across iterations #928
Merged
fix(ai): roll up RUN_FINISHED usage across iterations onto root span #961
Closed

Navigation Menu

Uh oh!

Uh oh!

feat(ai): emit cost + full usage on otel spans - #747

feat(ai): emit cost + full usage on otel spans#747
AlemTuzlak merged 2 commits into
TanStack:mainfrom
season179:feat/otel-full-usage-emission

season179 commented Jun 11, 2026•
edited by coderabbitai Bot
Loading

Uh oh!

coderabbitaiBot commented Jun 11, 2026•
edited
Loading

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Poem

❌ Failed checks (1 warning)

Uh oh!

nx-cloudBot commented Jun 16, 2026•
edited
Loading

Uh oh!

pkg-pr-newBot commented Jun 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

season179 commented Jun 11, 2026• edited by coderabbitai BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Tests

Summary by CodeRabbit

Uh oh!

coderabbitaiBot commented Jun 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Poem

❌ Failed checks (1 warning)

Uh oh!

nx-cloudBot commented Jun 16, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-newBot commented Jun 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

season179 commented Jun 11, 2026•
edited by coderabbitai Bot
Loading

coderabbitaiBot commented Jun 11, 2026•
edited
Loading

nx-cloudBot commented Jun 16, 2026•
edited
Loading