fix: record token usage for streaming generations

[TonsOfFun]

The bug

Any generation with stream: true reports zero tokens, zero cost, and no context-pressure estimate. Non-streaming is unaffected.

The visible symptoms are all downstream of one missing number: the dashboard shows Tokens 0 and Total Cost $0.00 beside a run that plainly made an API call, and the trace detail's Context Pressure bar does not render at all — its condition requires a span with non-zero tokens.

Why it happens

Three things had to be true at once for the usage to survive. None of them were.

1. There is no response body to read.api_prompt_execute consumes the stream and returns nil, so resolve_prompt — whose only source of usage is the response — has nothing to push onto usage_stack.

2. The provider was never asked for usage. Chat Completions emits usage on a final chunk, and only when the request sets stream_options: {include_usage: true}. Rather than hardcode the flag, this adds an api_stream_usage_parameters hook that defaults to {} — providers that report unconditionally, or not at all, are unaffected.

3. The response was built before the usage arrived. That final chunk lands aftercontent.done, which is exactly where the OpenAI chunk handler called process_prompt_finished. Completion is now deferred to stream_finished!, which the base provider calls once the stream drains. Its result is returned from resolve_prompt so the generation — and any tool-call recursion inside it — still runs exactly once.

There was also a silent conversion failure: Usage.from_provider_usage early-returns on anything that is not a Hash, and the stainless gems hand back model objects (OpenAI::Models::CompletionUsage). record_stream_usage now converts before delegating, ignores all-zero and nil payloads so the usage: null on ordinary content chunks does not add empty entries to a stack summed with reduce(:+), and reads usage before dereferencing choices.first because the usage chunk carries an empty choices array.

Verification

Against OpenRouter (anthropic/claude-sonnet-4.5):

	before	after
streaming probe	usage=nil	input=14 output=4 total=18
same prompt, non-streaming	input=14 output=4	unchanged
real 30-page enrichment batch	0/0	475/72

The streaming and non-streaming numbers now agree for an identical prompt, which is the property that matters.

Downstream, in a self-hosted dashboard: per-agent token counts and cost populate, and the Context Pressure bar renders.

Tests

test/providers/base_provider_stream_usage_test.rb covers usage on a final chunk, model-object conversion, the all-zero/nil guard, the Chat Completions opt-in, the empty default for other providers, and that a deferred completion runs process_prompt_finished exactly once.

⚠️Not run locally — the dummy app's encrypted credentials fail to decrypt in my checkout (ActiveSupport::MessageEncryptor::InvalidMessage, no master key), which breaks existing tests on a clean tree too. Relying on CI here.

Risk

Touches the shared streaming path for every provider. The deferral is opt-in per chunk handler: stream_completion_pending is only set by the OpenAI Chat handler, and stream_finished! is a no-op for handlers that still finish inline, so Anthropic/Gemini/Ollama/Bedrock behaviour is unchanged.

🤖 Generated with Claude Code