feat(runtime): trace per-request provider usage and cache attribution

[Astro-Han]

Problem

Maka cannot explain a provider cache miss at request level.

request-shape.ts computes hashes for the system prompt, provider options, tool schema, and history. The trace mainly persists a turn-start snapshot, while same-turn tool-loop hashes are computed but not retained as request records.

Usage has a similar gap. recordLlmCall records the aggregate after the send and tool loop, not one row per provider request or retry. A trace cannot reliably connect cache usage to the request that produced it.

In the Terminal-Bench v11 comparison in #1217, Kimi Code had provider-proxy telemetry for each request. Maka exposed cumulative cell usage. The Maka trace shows a stable turn-level prefix, but it cannot identify which request missed or which request segment changed.

#615 already proposes capturing the exact prepared request. This issue should add usage and cache telemetry to that capture path, not create another request snapshot.

Desired outcome

Record one telemetry row for every physical provider request attempt.

The row needs:

• turn, step, attempt, and request-capture reference;
• input, cache-read, cache-miss, cache-write, output, and reasoning tokens;
• whether cache fields came from the provider or were derived;
• status, finish reason, latency, and time to first token when available;
• hashes and sizes for the prepared request segments.

Segment comparison can derive the first changed cacheable segment. It does not need another persisted classification field.

Acceptance criteria

• Tool-loop requests, retries, failures, and interruptions have separate records.
• Each usage record links to the request captured by fix(runtime): RuntimeEvent ledger is recovery-grade (WAL), not source-of-truth — write-path truncation blocks counterfactual replay #615.
• Provider-reported and derived cache values remain distinguishable.
• Missing provider usage remains missing rather than being estimated from partial data.
• Complete per-request usage reconciles with the existing aggregate.
• Request detail lives in RunTrace or referenced artifacts; RuntimeEvent keeps only the aggregate and reference needed for replay.
• Tests cover Anthropic-compatible and OpenAI-compatible multi-step requests.
• Harbor exports can identify the first changed request segment without a provider proxy.

Related

• fix(runtime): RuntimeEvent ledger is recovery-grade (WAL), not source-of-truth — write-path truncation blocks counterfactual replay #615
• feat(benchmark): extend economic mechanism A/B testing beyond PR #340 active tool prune #481
• docs: Terminal-Bench 2.1 Maka vs Kimi Code v11 results #1217

[Nyvo-io]

I would like to claim #1268: per-request provider usage and cache attribution.

Planned scope:

• reuse the request-capture path from fix(runtime): RuntimeEvent ledger is recovery-grade (WAL), not source-of-truth — write-path truncation blocks counterfactual replay #615 rather than introducing a parallel snapshot mechanism;
• record one durable telemetry row per physical provider request attempt, including tool-loop steps, retries, failures, and interruptions;
• link each row to its request capture and preserve the distinction between provider-reported and derived cache fields;
• keep RuntimeEvent replay state bounded to the aggregate/reference while placing request detail in RunTrace or referenced artifacts;
• reconcile complete per-request usage with the existing aggregate without estimating missing provider fields;
• add focused Anthropic-compatible and OpenAI-compatible multi-step coverage, including Harbor export evidence for the first changed request segment.

I will first audit the existing request-shape, RunTrace, usage, retry, and provider adapter contracts and confirm the #615 boundary. The implementation will preserve current aggregate/public behavior, avoid provider-proxy assumptions, and keep protocol-specific normalization behind existing provider boundaries. I will wait for maintainer assignment before creating the implementation branch.

[Astro-Han]

Thanks for offering to take this. I had already started working on #1268 and have now opened #1277 with the implementation. Sorry I didn’t leave a note on the issue earlier to avoid the overlap.

The PR covers the request-capture artifact, per-attempt telemetry for tool loops, retries, failures, interruptions, and cancellation, cache provenance, aggregate trace references, and Harbor segment analysis. If you haven’t started yet, #1277 should cover the issue, so a separate implementation shouldn’t be necessary.