Use gateway spend logs for logical auto-router model pricing

[pitimon]

Phase 1 decision — 2026-08-20

Mode: Existing-system mode.

The owner selected a local Hermes cost-consumption phase. TokenTracker will consume only explicitly authoritative per-model cost already persisted in local Hermes state.db; it will not query LiteLLM spend logs, hold a gateway/admin credential, connect to gateway PostgreSQL, or infer ownership from route/time.

Problem

A logical routed model can be priced by TokenTracker through a generic litellm:fuzzy match even when Hermes has a more authoritative per-model amount. An auto-router must not be pinned to one child model because its selected model and cache rate can vary.

Confirmed evidence

• TokenTracker keeps the latest local queue row for (source, model, hour_start) and normally derives cost from token columns plus the model matcher (src/lib/local-api.js, src/lib/pricing/index.js).
• Hermes session_model_usage contains per-model estimated_cost_usd, actual_cost_usd, cost_status, cost_source, first/last timestamps, and token columns.
• The current local Hermes data has only unknown / none cost status with zero actual cost. This change is therefore forward-compatible plumbing, not proof that the current gateway has begun supplying authoritative spend.
• LiteLLM spend-log storage has fields suitable for a future independent collector (model_group, physical model, spend, client-scope fields), but the active Hermes virtual key receives HTTP 403 for /spend/logs; a management-credential collector is deliberately out of scope for this phase.

Feature: Consume Hermes authoritative cost locally

What: On a compatible Hermes schema, propagate a per-model row explicitly marked cost_status=actual into the local TokenTracker queue and prefer its actual_cost_usd over the local price matcher for that queue row.

Who: A local TokenTracker user running Hermes through a gateway or provider that already persists authoritative per-call cost in state.db.

In scope

• Read compatible cost columns from local session_model_usage.
• Carry authoritative cumulative cost deltas into the existing local half-hour queue.
• Expose non-secret hermes-actual, estimated, or mixed provenance in model-breakdown data.
• Use the same selected cost in summary, daily, hourly, model-breakdown, and plan-value inputs.
• Preserve current matching for old Hermes schemas, unknown, estimated, partial, or pre-existing queue data.

Out of scope

• LiteLLM /spend/logs, gateway database, export-file collectors, credentials, virtual-key aliases, request IDs, network calls, or direct gateway correlation.
• Treating a gateway-calculated cost as an upstream-provider invoice.
• Historical queue rewrites or retroactive replacement of estimates.
• Project-level authoritative attribution.

EARS acceptance criteria

1. [Optional] FR-001: Where Hermes lacks compatible cost columns or does not explicitly mark a per-model row actual, TokenTracker shall retain existing local cost estimation.
2. [Event-driven] FR-002: When a compatible Hermes row has cost_status=actual, TokenTracker shall add only its cumulative actual_cost_usd delta to the corresponding local half-hour model bucket and emit cost_provenance=hermes-actual.
3. [Unwanted] FR-003: If any task contributing to an aggregated Hermes model row lacks an explicit actual status, TokenTracker shall not label the partial amount authoritative.
4. [Ubiquitous] FR-004: Re-reading an unchanged Hermes state row shall not change queue cost, and a later cumulative increase shall contribute only the new delta.
5. [State-driven] FR-005: While a queue row has cost_provenance=hermes-actual, summary, daily, hourly, and model-breakdown cost shall use actual_cost_usd; the model shall not be added to fuzzy or unpriced diagnostics.
6. [Ubiquitous] FR-006: TokenTracker shall expose only fixed provenance labels; it shall not store or expose Hermes session IDs, prompts, request payloads, gateway credentials, or gateway identifiers.

Success criteria

• An actual-cost fixture supersedes an otherwise unpriced/fuzzy logical model with the exact persisted amount.
• A partial actual/unknown mixed-task fixture remains estimated.
• Second sync is idempotent; later actual cumulative growth is delta-only.
• Focused parser, API, dashboard-data, OpenWiki fact, typecheck, and full local CI tests pass.

Follow-up: gateway collection

A separate Hermes/gateway issue is required before current routed traffic can populate actual_cost_usd. It must choose a read-only transport and an explicit client-scope/correlation contract; no shared-gateway total may be attributed from logical route and time alone.

[pitimon]

Reopened: PR #185 added a local-only consumer for Hermes actual_cost_usd, but the motivating logical route still reports pricing_tier=litellm:fuzzy and cost_provenance=estimated because current Hermes rows contain no authoritative cost. It therefore does not resolve the gateway spend-log problem described here. The remaining work is to establish a secure, client-scoped authoritative gateway-cost source and reconciliation path.