Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/billed-usage-unit.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
---
'@tanstack/ai-event-client': minor
'@tanstack/ai': minor
'@tanstack/ai-fal': minor
'@tanstack/ai-grok': minor
'@tanstack/ai-openai': minor
'@tanstack/ai-byteplus': minor
'@tanstack/ai-cohere': minor
'@tanstack/ai-openrouter': minor
'@tanstack/ai-persistence': minor
---

Add a self-describing `billed` field to `TokenUsage` so billed quantities carry the unit they are counted in (#816). `usage.billed` is `{ quantity, unit }` with a `BillingUnit` union (`'seconds'`, `'units'`, `'images'`, `'tokens'`, ... open-ended). The deprecated `unitsBilled` / `durationSeconds` counts are still populated for backward compatibility. The fal adapters report `{ quantity, unit: 'units' }`, Grok video `{ quantity, unit: 'seconds' }`, the OpenAI/Grok/BytePlus duration-billed transcription paths `{ quantity, unit: 'seconds' }`, BytePlus Seedream images `{ quantity, unit: 'images' }`, BytePlus Seedance video `{ quantity, unit: 'tokens' }`, and Cohere/OpenRouter rerank `{ quantity, unit: 'units' }` (search units). Persistence sums `billed` when both reports use the same unit. `otelMiddleware` emits the pair as `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` span attributes.
2 changes: 1 addition & 1 deletion docs/adapters/grok.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -366,7 +366,7 @@ const extension = await generateVideo({

Both return the usual `{ jobId }` and are polled like any other Grok video job.

When the job completes, the adapter reports usage on the result: `usage.unitsBilled` carries the billed seconds of video and `usage.cost` the exact cost in USD, both as returned by the xAI API.
When the job completes, the adapter reports usage on the result: `usage.billed` carries the billed seconds of video (`{ quantity, unit: 'seconds' }`) and `usage.cost` the exact cost in USD, both as returned by the xAI API.

See [Video Generation](../media/video-generation) for the full jobs/polling flow, streaming mode, and the `useGenerateVideo` hook.

Expand Down
11 changes: 8 additions & 3 deletions docs/advanced/otel.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -79,7 +79,10 @@ Iteration spans are numbered (`#0`, `#1`, ...) in the order model calls are obse
| root / iteration | `gen_ai.usage.cache_read.input_tokens` | cached prompt tokens, when reported |
| root / iteration | `gen_ai.usage.cache_creation.input_tokens` | cache-write prompt tokens, when reported |
| root / iteration | `gen_ai.usage.reasoning.output_tokens` | reasoning/thinking tokens, when reported |
| root / iteration | `tanstack.ai.usage.duration_seconds` | duration-based billing (e.g. transcription), when reported |
| root / iteration | `tanstack.ai.usage.billed_quantity` | non-token billed quantity, when reported |
| root / iteration | `tanstack.ai.usage.billed_unit` | unit of the billed quantity (`seconds`, `units`, ...) |
| root / iteration | `tanstack.ai.usage.duration_seconds` | deprecated duration count; read `billed_quantity`/`billed_unit` instead |
| root / iteration | `tanstack.ai.usage.units_billed` | deprecated bare unit count; read `billed_quantity`/`billed_unit` instead |
| root / iteration | `tanstack.ai.usage.upstream_cost` | gateway upstream cost (e.g. OpenRouter), when reported |
| root / iteration | `tanstack.ai.usage.upstream_input_cost` | upstream input cost split, when reported |
| root / iteration | `tanstack.ai.usage.upstream_output_cost` | upstream output cost split, when reported |
Expand All@@ -92,7 +95,9 @@ Iteration spans are numbered (`#0`, `#1`, ...) in the order model calls are obse
| tool | `gen_ai.tool.type` | `function` |
| tool | `tanstack.ai.tool.outcome` | `success` / `error` |

Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (duration-based billing, the upstream cost split) are TanStack-namespaced.
Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (the billed quantity/unit pair, the upstream cost split, and the deprecated bare counts) are TanStack-namespaced.

For non-token billing (seconds of video or transcription, fal's endpoint units, ...), `tanstack.ai.usage.billed_quantity` and `tanstack.ai.usage.billed_unit` are emitted as a pair from `usage.billed`, so backends can label and aggregate media usage without knowing the provider. The deprecated `duration_seconds` / `units_billed` attributes carry the same quantities without the unit and remain emitted for backward compatibility.

### Metrics

Expand DownExpand Up@@ -234,7 +239,7 @@ Each media call produces one `CLIENT` span tagged with the activity's `gen_ai.op
| `generateTranscription` | `transcription` |
| `summarize` | `summarize` |

The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including `tanstack.ai.usage.units_billed` for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle. Non-streaming video is two calls, so the submit itself emits no span — the run opens once the provider accepts the job, and the `getVideoJobStatus()` poll that observes a terminal state ends it. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked.
The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including the `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` pair for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle. Non-streaming video is two calls, so the submit itself emits no span — the run opens once the provider accepts the job, and the `getVideoJobStatus()` poll that observes a terminal state ends it. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked.

`otelMiddleware` applies the same `spanNameFormatter`, `attributeEnricher`, `onBeforeSpanStart`, and `onSpanEnd` extension points to media spans — the span info is discriminated by `kind`, where media spans report `kind: 'generation'`. For a custom backend, implement the base `GenerationMiddleware` contract directly; its hooks (`onStart` / `onUsage` / `onFinish` / `onAbort` / `onError`) receive the `GenerationMiddlewareContext` and fire for every activity, chat included. The `GenerationMiddleware` types are exported from the package root, while the `otelMiddleware` value lives on the `@tanstack/ai/middlewares/otel` subpath so importing `@tanstack/ai` never requires the optional `@opentelemetry/api` peer.

Expand Down
9 changes: 5 additions & 4 deletions docs/config.json
Original file line numberDiff line numberDiff line change
Expand Up@@ -440,7 +440,7 @@
"label": "Audio Generation",
"to": "media/audio-generation",
"addedAt": "2026-04-23",
"updatedAt": "2026-08-04"
"updatedAt": "2026-08-08"
},
{
"label": "Image Generation",
Expand DownExpand Up@@ -479,7 +479,8 @@
{
"label": "Reranking",
"to": "rerank/rerank",
"addedAt": "2026-06-25"
"addedAt": "2026-06-25",
"updatedAt": "2026-08-19"
}
]
},
Expand DownExpand Up@@ -507,7 +508,7 @@
"label": "OpenTelemetry",
"to": "advanced/otel",
"addedAt": "2026-05-08",
"updatedAt": "2026-08-06"
"updatedAt": "2026-08-08"
}
]
},
Expand DownExpand Up@@ -823,7 +824,7 @@
"label": "Grok (xAI)",
"to": "adapters/grok",
"addedAt": "2026-04-15",
"updatedAt": "2026-08-18"
"updatedAt": "2026-08-19"
},
{
"label": "Groq",
Expand Down
7 changes: 4 additions & 3 deletions docs/media/audio-generation.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -199,9 +199,10 @@ interface AudioGenerationResult {
}
// Canonical TokenUsage (same shape as chat), present when the provider
// reports it (e.g. Gemini Lyria via generateContent). Usage-billed providers
// (fal) instead surface `usage.unitsBilled` — the real billed quantity read
// from fal's `x-fal-billable-units` result header. Multiply by the endpoint's
// unit price (fal pricing API) for the exact cost.
// (fal) instead surface `usage.billed` ({ quantity, unit: 'units' }) — the
// real billed quantity read from fal's `x-fal-billable-units` result header.
// Multiply the quantity by the endpoint's unit price (fal pricing API) for
// the exact cost.
usage?: TokenUsage
}
```
Expand Down
15 changes: 8 additions & 7 deletions docs/media/image-generation.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -705,7 +705,7 @@ interface ImageGenerationResult {
// Canonical TokenUsage (same shape as chat). Token-billed models also surface
// a per-modality breakdown on `promptTokensDetails` (e.g. text vs image input
// tokens for gpt-image-1). Usage-billed providers (fal) instead surface
// `usage.unitsBilled` — see the note below.
// `usage.billed` ({ quantity, unit }) — see the note below.
usage?: TokenUsage;
}

Expand All@@ -717,9 +717,9 @@ interface GeneratedImage {
```

> **Cost tracking (fal):** fal bills by usage-based units rather than tokens. The
> fal image adapter surfaces the real billed quantity as `usage.unitsBilled`
> (read from fal's `x-fal-billable-units` result header). Multiply it by the
> endpoint's unit price from
> fal image adapter surfaces the real billed quantity as `usage.billed` —
> `{ quantity, unit: 'units' }`, read from fal's `x-fal-billable-units` result
> header. Multiply the quantity by the endpoint's unit price from
> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` for the exact cost —
> no `fetch` interceptor needed.

Expand All@@ -733,9 +733,10 @@ const result = await generateImage({
prompt: "a serene mountain lake",
});

if (result.usage?.unitsBilled != null) {
const cost = result.usage.unitsBilled * unitPrice; // unitPrice from fal pricing API
console.log(`Billed ${result.usage.unitsBilled} units (~$${cost})`);
if (result.usage?.billed) {
const { quantity, unit } = result.usage.billed;
const cost = quantity * unitPrice; // unitPrice from fal pricing API
console.log(`Billed ${quantity} ${unit} (~$${cost})`);
}
```

Expand Down
23 changes: 14 additions & 9 deletions docs/media/video-generation.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -786,7 +786,7 @@ adapter.snapDuration(2.5); // 3 — clamped/rounded into range
adapter.snapDuration(99); // 15
```

Generated clips include an audio track. When the job completes, the adapter reports `usage.unitsBilled` (billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result.
Generated clips include an audio track. When the job completes, the adapter reports `usage.billed` (`{ quantity, unit: 'seconds' }` — billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result.

#### BytePlus (Seedance) Model Options

Expand All@@ -811,7 +811,7 @@ const { jobId } = await generateVideo({

Options are **model-specific and validated server-side**: Ark rejects an inapplicable field with a `400` instead of ignoring it. `service_tier` and `camera_fixed` are Seedance 1.x only, `frames` works on the 1.0-pro models, `draft` on 1.5-pro, `priority` on Seedance 2.5 and the 2.0 family, and `duration: -1` (let the model choose) on 2.5, 2.0 and 1.5-pro. Durations are 4–30s on Seedance 2.5, 4–15s on the 2.0 family, 4–12s on 1.5-pro and 2–12s on the 1.0-pro models.

**Seedance video URLs expire 24 hours after the task completes** (the task record is kept for seven days), so persist the bytes rather than the link. See the [BytePlus adapter](../adapters/byteplus#video-generation-seedance) for the full option table.
**Seedance video URLs expire 24 hours after the task completes** (the task record is kept for seven days), so persist the bytes rather than the link. See the [BytePlus adapter](../adapters/byteplus#video-generation-seedance) for the full option table. Completed jobs report `usage.billed` as `{ quantity, unit: 'tokens' }` (Seedance bills output tokens only).

##### Porting a Seedance call between providers

Expand DownExpand Up@@ -921,19 +921,24 @@ interface VideoUrlResult {
jobId: string;
url: string; // URL to download/stream the video
expiresAt?: Date; // When the URL expires
// Usage for the completed generation, when the adapter reports it. fal
// populates `usage.unitsBilled` from its `x-fal-billable-units` header.
// Usage for the completed generation, when the adapter reports it. The
// billed quantity is self-describing: fal reports
// `usage.billed = { quantity, unit: 'units' }` (from its
// `x-fal-billable-units` header), Grok Imagine reports
// `{ quantity, unit: 'seconds' }`.
usage?: TokenUsage;
}
```

> **Cost tracking (fal):** fal bills media generation by usage-based units
> rather than tokens. The fal adapters surface the real billed quantity as
> `usage.unitsBilled` (denominated in the endpoint's priced unit). Combine it
> with the endpoint's unit price from
> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to compute the exact
> cost (`unitsBilled * unitPrice`). The same `usage.unitsBilled` is surfaced
> on image, audio, speech, and transcription results.
> `usage.billed` — `{ quantity, unit: 'units' }`, where `'units'` marks fal's
> endpoint-defined priced unit. Combine the quantity with the endpoint's unit
> price from `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to
> compute the exact cost (`billed.quantity * unitPrice`). The same
> `usage.billed` is surfaced on image, audio, speech, and transcription
> results. (The deprecated bare count `usage.unitsBilled` is still populated
> for backward compatibility.)

### Model Variants

Expand Down
7 changes: 4 additions & 3 deletions docs/reference/interfaces/RerankResult.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -90,6 +90,7 @@ usage: TokenUsage;
Defined in: [packages/ai/src/types.ts:2085](https://github.com/TanStack/ai/blob/main/packages/ai/src/types.ts#L2085)

Usage for the request. Rerank typically bills in provider-defined "search
units" (`usage.unitsBilled`) rather than tokens. Some providers (e.g.
OpenRouter) may also report `totalTokens` and `cost`; Cohere reports only
search units and leaves the token counts at 0.
units" (`usage.billed = { quantity, unit: 'units' }`) rather than tokens. Some
providers (for example OpenRouter) may also report `totalTokens` and `cost`.
Cohere reports only search units and leaves the token counts at 0. The
deprecated `unitsBilled` field is still populated for compatibility.
4 changes: 2 additions & 2 deletions docs/reference/interfaces/VideoUrlResult.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -82,5 +82,5 @@ Defined in: [packages/ai/src/types.ts:2469](https://github.com/TanStack/ai/blob/
**`Experimental`**

Usage information for the completed generation, when the adapter can report
it. For usage-based providers (e.g. fal) this carries `unitsBilled` — the
real billed quantity so consumers can compute exact cost.
it. For usage-based providers (for example fal) this carries `billed`, the
real billed quantity paired with its unit, so consumers can compute exact cost.
13 changes: 9 additions & 4 deletions docs/rerank/rerank.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -164,9 +164,10 @@ interface RerankResult<TDocument = string> {
ranking: Array<{ index: number; score: number; document: TDocument }>
// The documents reordered by relevance (ranking.map(r => r.document)).
rerankedDocuments: Array<TDocument>
// Rerank typically bills in provider "search units" (usage.unitsBilled).
// Some providers (e.g. OpenRouter) also report totalTokens and cost; Cohere
// reports only search units and leaves token counts at 0.
// Rerank typically bills in provider "search units"
// (usage.billed = { quantity, unit: 'units' }). Some providers (for example
// OpenRouter) also report totalTokens and cost. Cohere reports only search
// units and leaves token counts at 0.
usage: TokenUsage
}
```
Expand DownExpand Up@@ -295,7 +296,11 @@ const result = await rerank({
{
name: 'usage-logger',
onUsage: (_ctx, usage) => {
console.log('search units billed:', usage.unitsBilled)
if (usage.billed) {
console.log(
`search units billed: ${usage.billed.quantity} ${usage.billed.unit}`,
)
}
},
},
],
Expand Down
11 changes: 6 additions & 5 deletions examples/ts-react-media/src/components/ImageGenerator.tsx
Original file line numberDiff line numberDiff line change
Expand Up@@ -354,12 +354,13 @@ function ImageModelCard({
className="w-full h-auto"
/>
</div>
{result?.usage?.unitsBilled != null && (
{result?.usage?.billed && (
<p className="text-xs text-gray-500">
Billed {result.usage.unitsBilled}{' '}
{model.provider === 'fal' ? 'fal ' : ''}unit
{result.usage.unitsBilled === 1 ? '' : 's'} — multiply by the
endpoint unit price for USD cost
Billed {result.usage.billed.quantity}{' '}
{result.usage.billed.unit === 'units'
? `fal unit${result.usage.billed.quantity === 1 ? '' : 's'}`
: result.usage.billed.unit}{' '}
— multiply by the endpoint unit price for USD cost
</p>
)}
</>
Expand Down
4 changes: 2 additions & 2 deletions examples/ts-react-media/src/components/SeedanceStudio.tsx
Original file line numberDiff line numberDiff line change
Expand Up@@ -1238,10 +1238,10 @@ export default function SeedanceStudio({
value={formatElapsed(finishedAt - startedAt)}
/>
)}
{billing && (
{billing?.totalTokens != null && (
<Meta
label="Billed tokens"
value={`${billing.unitsBilled ?? billing.totalTokens}`}
value={`${billing.totalTokens}`}
/>
)}
</dl>
Expand Down
30 changes: 23 additions & 7 deletions examples/ts-react-media/src/components/VideoGenerator.tsx
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import { Film, Loader2, Shuffle, Upload, Wand2, X } from 'lucide-react'
import { useGenerateVideo } from '@tanstack/ai-react'
import type { BilledUsage } from '@tanstack/ai'
import type { VideoModel, VideoMode } from '@/lib/models'
import type { AttachedMedia } from '@/lib/media'
import type { MediaPrompt, MediaPromptPart } from '@tanstack/ai/client'
Expand DownExpand Up@@ -57,6 +58,22 @@ function buildVideoPrompt(
return parts.length === 1 ? request.prompt : parts
}

/**
* Human label for a billed quantity, driven by the unit the adapter reported —
* no guessing from provider identity or cost presence.
*/
function describeBilled({ quantity, unit }: BilledUsage): string {
const plural = quantity === 1 ? '' : 's'
switch (unit) {
case 'seconds':
return `${quantity} second${plural} of video`
case 'units':
return `${quantity} fal unit${plural}`
default:
return `${quantity} ${unit}`
}
}

export default function VideoGenerator({
initialImageUrl,
}: VideoGeneratorProps) {
Expand DownExpand Up@@ -559,16 +576,15 @@ function VideoModelCard({
{billing?.cost != null ? (
<p className="text-xs text-gray-500">
Billed ${billing.cost.toFixed(3)}
{billing.unitsBilled != null
? ` for ${billing.unitsBilled} second${billing.unitsBilled === 1 ? '' : 's'} of video`
: ''}
{billing.billed ? ` for ${describeBilled(billing.billed)}` : ''}
</p>
) : (
billing?.unitsBilled != null && (
billing?.billed && (
<p className="text-xs text-gray-500">
Billed {billing.unitsBilled} fal unit
{billing.unitsBilled === 1 ? '' : 's'} — multiply by the
endpoint unit price for USD cost
Billed {describeBilled(billing.billed)}
{billing.billed.unit === 'units'
? ' — multiply by the endpoint unit price for USD cost'
: ''}
</p>
)
)}
Expand Down
Loading
Loading