Skip to content

feat(metering): make ModelCallAttempt the metering source of truth - #1755

Merged
Astro-Han merged 5 commits into
apache:mainfrom
ARE404:are404/feat-canonical-metering-seam
Aug 1, 2026
Merged

feat(metering): make ModelCallAttempt the metering source of truth#1755
Astro-Han merged 5 commits into
apache:mainfrom
ARE404:are404/feat-canonical-metering-seam

Conversation

@ARE404

@ARE404ARE404 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Implements the metering half of #1679, on top of the ModelCallAttempt contract merged in #1687.

Landed as one PR rather than two, per the discussion on the issue: PR 2 alone was independently reviewable but never independently mergeable — it would have left a release window where either the Usage surface stops accruing, or two independent meters run side by side. Three commits, walkable in order.

Commits

1. feat(runtime): emit canonical ModelCallAttempt from the provider seam

ProviderRequestTracker gains one grouped optional accounting input — grouped rather than several independent fields because a record without session, run, and call kind is unattributable, and it spares the capture-only paths and their tests from fabricating identity.

  • Settlement never throws.finalize runs inside the stream's pull handler, so a rejection there reaches controller.error and fails an otherwise-complete model response. The dispatch-time gate is assertReady, checked before the provider is called and alongside the existing capture gate rather than as a second one. A sink failure after dispatch means the call happened and was billed but went unrecorded — reported, not raised.
  • Cancellation and settlement are separate events. An abort carrying no usage records provisionally without closing the attempt, so a finish arriving afterwards still settles it. Both writes share one attemptId and dedupe on that key keeping the last, so a cancelled call that really consumed tokens is no longer frozen as permanently token-less.
  • Retries are attempts of one logical call.logicalCallId is assigned per step and reused across retries.

Cost resolves at settlement time and carries the rates it was computed against, so a stored figure stays auditable when pricing later changes. An unresolvable price records costBasis: 'unpriced' with no amount — never zero, which is reserved for calls that genuinely cost nothing.

2. feat(core): project Usage aggregates from the canonical attempt ledger

Pure aggregation over ModelCallAttempt serving summary / buckets / logs. totalCostUsd sums only records whose price was resolvable, and every result carries the coverage that qualifies it — the old schema had no way to say "this call cost something we could not price", so it stored zero, and unpriced spend was indistinguishable from free. interrupted projects to aborted, not error, so user cancellations do not inflate the error rate.

3. feat(metering): read Usage from the canonical ledger and drop the send-level meter

The atomic one — the authority moves onto the canonical ledger and the writer it replaces comes out in the same commit.

Where records land. The AgentRun stream is the durable log (model_call_attempt_recorded, durable: true). It is not the read model: readEvents(sessionId, runId) answers "what happened in this run", and no Usage question is shaped that way. So each attempt also lands in a new usage_model_call_attempts table — beside usage_llm_calls, not inside it, since that schema cannot express usageBasis or costBasis.

The read path sums two sources. The old table is frozen and never migrated, and still receives the semantic_compact and history_compact calls that have not been routed through the seam yet. Every LLM-sourced result therefore carries UsageProvenance: the coverage of its canonical half, how many rows came from the frozen table, and how many stored records failed to decode. legacyRecords reaches zero on its own as the old table ages out of the queried range.

What comes out. The terminal recordLlmCall in the streaming backend and the usage_recorded RunTraceEvent. token_usage SessionEvents and the RuntimeEvent per-turn aggregates stay — they feed replay and recovery and are not accounting.

Two things worth a closer look in review

The deletion nearly took non-metering evidence with it. That record was also the only durable home for the send's terminal context diagnostics. The exhausted and aborted paths emit no token_usage SessionEvent, so their compaction decisions, their final request shape, and the accumulated usage of the steps that did complete would have disappeared. They move to a send_diagnostics_recorded run-trace event that carries no cost and meters nothing. The request-shape hashes needed no rescue — model_stream_started already carries step 0's set, and the new event carries the final shape a same-turn tool load produces. Flagging it because it falls outside what "delete the two writers" anticipated.

The Desktop runs a second, complete metering stack against its own store, write and read. Deleting the send-level writer globally would have stopped its Usage page accruing, so it reads and writes through the same ledger. This is wiring, not authority — nothing here gives the embedded writer ownership, election, or admission, and #1167's M5 can still remove it wholesale.

Also: the end-to-end acceptance caught two gaps in the seam from commit 1. Canonical records carried neither connectionSlug nor the connection's provider type, so they were attributable to a provider and model but not to a connection — and the SDK's raw moonshot.chat would have split one provider across two bucket keys against the historical rows.

Known limits, stated rather than implied

Two things this change does not deliver, called out so the guarantees in the code are not read as broader than they are:

  • Recovery has a residual window. The pending marker is written before the projection is attempted, so it survives a crash rather than depending on an error handler running. A process that dies between committing to the authority and writing that marker still leaves a record the Usage table will not learn about — nothing sweeps the whole stream. A full re-projection pass would close it; it is not implemented here.
  • The Desktop Usage page does not consume any of this yet. The renderer still calls settings.usageStats(), the session-derived aggregation, as it did before this PR. The canonical summary/bucket/log endpoints are wired and tested but have no renderer consumer, so per-attempt retry, abort, and provenance semantics are not visible there, and the Daily Review archives a merged total without carrying its provenance into the saved summary. Surfacing them is [RFC] Session Inspector: a per-session runtime event trace, replay, and cost-attribution surface #1625's work.

Not in scope

semantic_compact routing stays a separate PR, as agreed — it is new wiring rather than a re-route, and the merged read path keeps those calls counted in the meantime. Same for history_compact: routing it now would have created exactly the double write this PR removes.

Verification

Acceptance is end-to-end, against a real provider wire: execution-model-composition.test.ts reads the main usage row back through the coordinator from the canonical ledger — provider call → attempt record → projection → Usage surface.

Locally on the rebased branch: @maka/core 1267/1267, @maka/storage 928/928, @maka/runtime-host 435/435, @maka/runtime 2877/2891 (the remainder are four Grep/filesystem tests that fail in my shell because rg is a function rather than a binary here). All packages typecheck.

Closes part of #1679.

🤖 Generated with Claude Code

@ARE404
ARE404force-pushed the are404/feat-canonical-metering-seam branch from e24cb87 to be9a708CompareAugust 1, 2026 05:45
@Astro-Han

Copy link
Copy Markdown
Contributor

I reviewed the severity again. One finding remains blocking, and two localized P2 findings are supported by the current code.

P0

None.

P1

The Usage ledger is an independent write, not a recoverable read model.

execution-model-composition.ts and session-stream.ts write each ModelCallAttempt independently to the AgentRun stream and usage_model_call_attempts through Promise.allSettled. The code describes AgentRun as the durable log of record, but nothing replays or reconciles model_call_attempt_recorded events into the Usage table.

If the AgentRun append succeeds and the Usage-ledger write fails, the billed call is permanently missing from Usage. The reverse leaves the stated log of record incomplete. The Host requests a drain after a Usage-ledger failure but cannot recover the missing call. Desktop swallows the failure and continues dispatching.

This makes the Usage table a second source of truth. It also leaves out the rebuild/checkpoint behavior agreed in #1679. Since this PR establishes canonical accounting, I consider this blocking.

The required invariant is that a ModelCallAttempt is committed to one durable authority and the Usage table can be rebuilt from it through an idempotent projection.

P2

Legacy and canonical hourly buckets use incompatible keys.

model-call-usage-projection.ts uses an ISO-hour key such as 2025-06-15T15, while the legacy stores use an epoch-hour key such as 486015. mergeUsageBuckets merges only identical keys. A query containing legacy and canonical calls from the same hour therefore returns two buckets instead of one.

Both sources should use the same bucket-key function, with a contract test covering a mixed-source hourly query.

Unpriced canonical calls still appear as $0 in individual log rows.

projectModelCallUsageLogs maps an unpriced attempt to costUsd: 0, and UsageLogRow has no row-level costBasis. A genuinely free call and a call whose price is unknown are therefore indistinguishable in the log. Page-level coverage does not identify which row is unpriced.

The log contract should carry costBasis, with costUsd absent or null for an unpriced row.

P3

None.

The projection and merge tests cover the main successful paths, but they do not cover one-sided persistence failure, mixed-source hourly buckets, or mixed free/unpriced log rows. Those tests should accompany the fixes above.

ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 1, 2026
Addresses the review on apache#1755.
**P1 — the Usage table was an independent write, not a recoverable read model.**
The finding is right, and it undercut the claim the previous commit made about
itself. Both hosts wrote each `ModelCallAttempt` to the AgentRun stream and to
`usage_model_call_attempts` through `Promise.allSettled`, with nothing ever
replaying one into the other. Calling the stream "the log of record" does not
make it one: either write could land alone, and the Host's answer was to request
a drain it could not recover from while the Desktop swallowed the failure and
kept dispatching. That is two sources of truth, which is the thing this whole
change set exists to remove.
There is now one commit point. The AgentRun append is awaited first and
`AgentRun.recordModelCallAttempt` rejects on failure instead of resolving, so a
caller can tell whether the authority actually holds the record; the ledger is
written only after it does. Rejecting is safe because the seam still swallows it
— settlement runs inside the stream's `pull` handler, and a completed, billed
response must never be failed by its own bookkeeping.
A failed projection is recoverable. The run is marked in
`usage_model_call_reprojection`, and the Usage read path re-derives marked runs
from the stream before answering, bounded per query so a backlog drains across
reads instead of inside one. The projection is idempotent — it upserts on
`attemptId` — so repairing a run the table already holds is a no-op, and one
undecodable event does not block the rest of its run.
The marker is an optimization, not the correctness argument. Losing it costs a
targeted repair, not the records: the authority holds every attempt, so a full
re-projection recovers a run whose marker was never written. What a pass cannot
repair is reported as `provenance.pendingRepairs` rather than silently missing
from the totals — spend that is recoverable is a different claim from spend that
is counted.
The pre-dispatch gate moves with it. It now keys off the authority alone: a
stale projection is recoverable and must not block a send, but an authority that
cannot accept the record means the next dispatch produces spend nothing will
ever hold.
**P2 — one bucket-key function.**
The two sources derived "the same hour" differently — an epoch-hour ordinal
against an ISO hour — so `mergeUsageBuckets` saw two keys and split one hour in
half without failing anywhere. Both now call `usageBucketKey`, and a
mixed-source hourly query is a contract test. `day` happened to agree already;
that it did was luck, not design.
**P2 — a log row keeps its cost basis.**
`projectModelCallUsageLogs` mapped an unpriced attempt to `costUsd: 0`, so a
genuinely free call and a call whose price could not be resolved were
indistinguishable per row — page-level coverage says how many were unpriced, not
which. This reproduced, at row granularity, exactly the ambiguity the coverage
breakdown was added to remove. `UsageLogRow` and the wire projection now carry
`costBasis`, `costUsd` is absent for an unpriced row, and the codec rejects a
row that claims both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404
ARE404force-pushed the are404/feat-canonical-metering-seam branch from 039921c to 46b32a1CompareAugust 1, 2026 10:47
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 1, 2026
Addresses the review on apache#1755.
**P1 — the Usage table was an independent write, not a recoverable read model.**
The finding is right, and it undercut the claim the previous commit made about
itself. Both hosts wrote each `ModelCallAttempt` to the AgentRun stream and to
`usage_model_call_attempts` through `Promise.allSettled`, with nothing ever
replaying one into the other. Calling the stream "the log of record" does not
make it one: either write could land alone, and the Host's answer was to request
a drain it could not recover from while the Desktop swallowed the failure and
kept dispatching. That is two sources of truth, which is the thing this whole
change set exists to remove.
There is now one commit point. The AgentRun append is awaited first and
`AgentRun.recordModelCallAttempt` rejects on failure instead of resolving, so a
caller can tell whether the authority actually holds the record; the ledger is
written only after it does. Rejecting is safe because the seam still swallows it
— settlement runs inside the stream's `pull` handler, and a completed, billed
response must never be failed by its own bookkeeping.
A failed projection is recoverable. The run is marked in
`usage_model_call_reprojection`, and the Usage read path re-derives marked runs
from the stream before answering, bounded per query so a backlog drains across
reads instead of inside one. The projection is idempotent — it upserts on
`attemptId` — so repairing a run the table already holds is a no-op, and one
undecodable event does not block the rest of its run.
The marker is an optimization, not the correctness argument. Losing it costs a
targeted repair, not the records: the authority holds every attempt, so a full
re-projection recovers a run whose marker was never written. What a pass cannot
repair is reported as `provenance.pendingRepairs` rather than silently missing
from the totals — spend that is recoverable is a different claim from spend that
is counted.
The pre-dispatch gate moves with it. It now keys off the authority alone: a
stale projection is recoverable and must not block a send, but an authority that
cannot accept the record means the next dispatch produces spend nothing will
ever hold.
**P2 — one bucket-key function.**
The two sources derived "the same hour" differently — an epoch-hour ordinal
against an ISO hour — so `mergeUsageBuckets` saw two keys and split one hour in
half without failing anywhere. Both now call `usageBucketKey`, and a
mixed-source hourly query is a contract test. `day` happened to agree already;
that it did was luck, not design.
**P2 — a log row keeps its cost basis.**
`projectModelCallUsageLogs` mapped an unpriced attempt to `costUsd: 0`, so a
genuinely free call and a call whose price could not be resolved were
indistinguishable per row — page-level coverage says how many were unpriced, not
which. This reproduced, at row granularity, exactly the ambiguity the coverage
breakdown was added to remove. `UsageLogRow` and the wire projection now carry
`costBasis`, `costUsd` is absent for an unpriced row, and the codec rejects a
row that claims both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took another pass at the current head and revised the severity. I do not think these are blocking, but a few gaps remain.

  1. P2: the accounting gate checks the wrong failure flag

    recordModelCallAttempt sets accountingAuthorityFailed when the authority append fails, but assertModelCallAccountingReady checks telemetryDrainRequested instead (execution-model-composition.ts:320-341). accountingAuthorityFailed is never read, so later provider calls can still dispatch after the authority has stopped accepting records.

    This only affects the authority-failure path, so I would not rate it P1. It is also a small fix in the failure-containment path and is worth correcting here.

  2. P2: reprojection still depends on the pending marker

    If the AgentRun append succeeds and the process exits before the Usage projection or markRunPendingReprojection, the authority has the event but the repair path cannot find it. repairPendingModelCallProjections only iterates marked runs (model-call-ledger.ts:256-280).

    The PR text says the marker is an optimization and that a full reprojection recovers a missing marker, but I could not find that sweep in the current code. Either add it or narrow the stated recovery guarantee.

  3. P2: the Desktop Usage page still reads the session-derived path

    Settings still calls window.maka.settings.usageStats() (settings-surface.tsx:213-224), which routes to the session token_usage aggregation (usage-ipc-main.ts:79-81, usage-stats-store.ts:54-117). The new canonical summary, bucket, and log endpoints have no renderer consumer.

    This does not regress the existing page, but it means Desktop does not receive the per-attempt retry, failure, abort, or provenance semantics added here. Missing prices also continue to render as $0.00, with no visible distinction for unpriced, partial, unreadable, or pending data.

  4. P2: repair clears the marker after an unreadable authority event

    modelCallAttemptsFromRunEvents reports unreadableEvents, but the repair path projects the readable rows and clears the pending marker without preserving that count (model-call-ledger.ts:271-275). The unreadable call then disappears from both the totals and the pending-repair provenance.

  5. P3: Daily Review can archive an incomplete total without a qualifier

    Daily Review reads the merged ledger but does not repair pending runs or expose pending and unreadable provenance in the saved summary (daily-review-main.ts:81-115). A later repair will not update the archived review.

None of these needs to block the PR on its own. I would fix the flag mix-up in this patch because it is local and directly affects the new accounting gate. The other items can be follow-ups if the PR description is updated to match the guarantees the code provides today.

ARE404and others added 5 commits August 2, 2026 01:33
First half of the metering work in apache#1679: the provider request seam now
produces the canonical accounting record landed in apache#1687. The old writers are
untouched here and come out in the same PR before it opens.
`ProviderRequestTracker` gains an optional `accounting` input. It is one unit
rather than several independent fields because a `ModelCallAttempt` without
session, run, and call kind is unattributable — there is no useful state where
half of it is wired. Absent, the tracker stays purely diagnostic, which is what
the capture-only paths and their tests rely on.
Three behaviours the review on apache#1679 called for:
- **Settlement never throws.** `finalize` runs inside the stream's `pull`
handler, so a rejection there reaches `controller.error` and fails an
otherwise-complete model response. The dispatch-time gate is `assertReady`,
checked before the provider is called and alongside the existing capture gate
rather than as a second one. A sink failure after dispatch means the call
happened and was billed but went unrecorded, which is reported rather than
raised.
- **Cancellation and settlement are separate events.** An abort carrying no
usage records provisionally without closing the attempt, so a `finish`
arriving afterwards still settles it. Both writes share one `attemptId` and
the record dedupes on that key keeping the last, so a cancelled call that
really consumed tokens is no longer frozen as permanently token-less.
- **Retries are attempts of one logical call.** `logicalCallId` is assigned per
step and reused across retries, so the grouping is explicit in the record
instead of reconstructed from `(traceId, step)` by every consumer.
Cost resolves at settlement time and carries the rates it was computed against,
so a stored figure stays auditable when pricing later changes. An unresolvable
price records `costBasis: 'unpriced'` with no amount — never zero, which is
reserved for calls that genuinely cost nothing. `usageBasis` separately reports
whether the provider returned usage at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second commit of the metering slice in apache#1679. Pure aggregation over
`ModelCallAttempt`, serving the shapes the Usage authority reads today —
`summary`, `buckets`, `logs` — so the read path can move off the per-send
`LlmCallRecord` table in the next commit. Nothing is wired yet.
One behavioural difference from the old table is deliberate, and is the reason
this projection exists rather than a schema-compatible one. `totalCostUsd` sums
only records whose price was resolvable, and every result carries the coverage
that qualifies it. The old schema had no way to say "this call cost something
we could not price", so it stored zero, and unpriced spend was indistinguishable
from a free call. A total presented without its coverage repeats that claim.
Two mapping decisions worth stating:
- `interrupted` projects to `aborted`, not `error`. Both mean the call stopped
short without the provider reporting a failure, and folding it into `error`
would inflate the error rate with user cancellations.
- Log rows carry `logicalCallId` as `callId` and keep session and turn
attribution, so a row remains traceable back to the conversation that caused
it rather than only to a model.
Selection dedupes on `attemptId` before aggregating, so a re-appended
settlement — the abort-then-late-usage path from the previous commit — counts
once.
Tool telemetry is untouched: `toolLogs` reads tool invocations, which this
ledger does not describe.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…d-level meter
Third and last commit of the metering slice in apache#1679, and the atomic one: the
Usage authority moves onto `ModelCallAttempt` and the writer it replaces comes
out in the same change, so no commit leaves two independent meters running.
**Where canonical records land.** The AgentRun stream is the durable log —
`model_call_attempt_recorded`, `durable: true`, because a record lost to a
crashed flush is spend nothing else can reconstruct. It is not the read model:
`AgentRunStore.readEvents(sessionId, runId)` answers "what happened in this
run", and no Usage question is shaped that way. So each attempt also lands in
`usage_model_call_attempts`, a new table beside the old one rather than inside
it — `usage_llm_calls` cannot express `usageBasis` or `costBasis`, and writing
canonical records through that schema is the dishonest-zero problem again.
**The read path sums two sources.** The old table is frozen, never migrated, and
still receives the `semantic_compact` and `history_compact` calls that have not
been routed through the canonical seam yet. Every LLM-sourced result therefore
carries `UsageProvenance`: the coverage of its canonical half, how many rows came
from the frozen table, and how many stored records failed to decode. A total on
the wire without that cannot be read honestly, so it is part of the wire.
`legacyRecords` reaches zero on its own as the old table ages out of the range.
**What comes out.** The terminal `recordLlmCall` in the streaming backend and the
`usage_recorded` RunTraceEvent. Both measured the same provider requests the
seam now settles per request instead of per send. `token_usage` SessionEvents and
the RuntimeEvent per-turn aggregates stay — they feed replay and recovery and are
not accounting.
**What the deletion nearly took with it.** That record was also the only durable
home for the send's terminal context diagnostics. The exhausted and aborted paths
emit no `token_usage` event, so their compaction decisions, their final request
shape, and the accumulated usage of the steps that did complete would have gone
with it. They move to a `send_diagnostics_recorded` run-trace event, which
carries no cost and meters nothing. The request-shape hashes needed no rescue —
`model_stream_started` already carries them for step 0, and the new event carries
the final shape a same-turn tool load produces.
Two gaps the end-to-end acceptance found in the seam from the first commit:
canonical records carried neither `connectionSlug` nor the connection's provider
type, so they were attributable to a provider and model but not to a connection,
and `moonshot.chat` would have split one provider across two bucket keys against
the historical rows.
The Desktop runs a second, complete metering stack against its own store. Deleting
the send-level writer globally would have stopped its Usage page accruing, so it
reads and writes through the same ledger. That is wiring, not authority: nothing
here gives the embedded writer ownership, election, or admission.
Acceptance is end-to-end and now green against a real provider wire: provider
call → attempt record → projection → Usage surface.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Addresses the review on apache#1755.
**P1 — the Usage table was an independent write, not a recoverable read model.**
The finding is right, and it undercut the claim the previous commit made about
itself. Both hosts wrote each `ModelCallAttempt` to the AgentRun stream and to
`usage_model_call_attempts` through `Promise.allSettled`, with nothing ever
replaying one into the other. Calling the stream "the log of record" does not
make it one: either write could land alone, and the Host's answer was to request
a drain it could not recover from while the Desktop swallowed the failure and
kept dispatching. That is two sources of truth, which is the thing this whole
change set exists to remove.
There is now one commit point. The AgentRun append is awaited first and
`AgentRun.recordModelCallAttempt` rejects on failure instead of resolving, so a
caller can tell whether the authority actually holds the record; the ledger is
written only after it does. Rejecting is safe because the seam still swallows it
— settlement runs inside the stream's `pull` handler, and a completed, billed
response must never be failed by its own bookkeeping.
A failed projection is recoverable. The run is marked in
`usage_model_call_reprojection`, and the Usage read path re-derives marked runs
from the stream before answering, bounded per query so a backlog drains across
reads instead of inside one. The projection is idempotent — it upserts on
`attemptId` — so repairing a run the table already holds is a no-op, and one
undecodable event does not block the rest of its run.
The marker is an optimization, not the correctness argument. Losing it costs a
targeted repair, not the records: the authority holds every attempt, so a full
re-projection recovers a run whose marker was never written. What a pass cannot
repair is reported as `provenance.pendingRepairs` rather than silently missing
from the totals — spend that is recoverable is a different claim from spend that
is counted.
The pre-dispatch gate moves with it. It now keys off the authority alone: a
stale projection is recoverable and must not block a send, but an authority that
cannot accept the record means the next dispatch produces spend nothing will
ever hold.
**P2 — one bucket-key function.**
The two sources derived "the same hour" differently — an epoch-hour ordinal
against an ISO hour — so `mergeUsageBuckets` saw two keys and split one hour in
half without failing anywhere. Both now call `usageBucketKey`, and a
mixed-source hourly query is a contract test. `day` happened to agree already;
that it did was luck, not design.
**P2 — a log row keeps its cost basis.**
`projectModelCallUsageLogs` mapped an unpriced attempt to `costUsd: 0`, so a
genuinely free call and a call whose price could not be resolved were
indistinguishable per row — page-level coverage says how many were unpriced, not
which. This reproduced, at row granularity, exactly the ambiguity the coverage
breakdown was added to remove. `UsageLogRow` and the wire projection now carry
`costBasis`, `costUsd` is absent for an unpriced row, and the codec rejects a
row that claims both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…nest
Second review round on apache#1755.
**The accounting gate was dead code.** `recordModelCallAttempt` set
`accountingAuthorityFailed`, and `assertModelCallAccountingReady` checked
`telemetryDrainRequested` — a flag that tracks the frozen legacy table, which no
longer meters main sends at all. Nothing read the new flag, so a host whose
authority had stopped accepting records kept dispatching provider calls it could
never account for. The gate the previous commit introduced never actually
closed; it does now.
**The pending marker is written before the projection, not after it fails.** A
marker written from a catch block cannot cover the case where the catch never
runs — the process exiting between the authority append and the projection —
which left a committed record this table would never learn about. Marking first
turns it into an intent record: a crash anywhere after it still leaves a run the
repair finds.
That does not close the window entirely, and the docs no longer say it does. A
process that dies between committing to the authority and writing the marker
still leaves a record Usage will not see, because nothing sweeps the whole
stream. The claim that "a full re-projection recovers a run whose marker was
never written" described a sweep that was never implemented; the comment and the
evidence-spine doc now state the real limit instead.
**An unreadable authority event no longer disappears with the marker.** The
repair decoded what it could, cleared the run, and dropped the undecodable
count — so a real call fell out of the totals and out of the pending count in
the same step, leaving nothing to say it existed. Clearing the marker is still
right (it will not decode next pass either, and holding the run would stall
every later repair behind it), so the count travels out on the result and folds
into `provenance.unreadableRecords` alongside stored rows that will not decode.
Two review items are deliberately left as follow-ups, and the PR description
says so rather than implying they are covered: the Desktop Usage page still
reads the session-derived `settings.usageStats` path and has no consumer for the
canonical endpoints, and the Daily Review archives a merged total without
carrying its provenance into the saved summary.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ARE404
ARE404force-pushed the are404/feat-canonical-metering-seam branch from 46b32a1 to 8cbcd5dCompareAugust 1, 2026 17:43
@ARE404

Copy link
Copy Markdown
ContributorAuthor

All five verified against the code — all five hold. Rebased onto current main and pushed 8cbcd5d6.

1 — the gate checked the wrong flag. Confirmed, and worse than "a small fix": accountingAuthorityFailed was never read, so the gate the last commit introduced never closed at all. Fixed to read its own flag. telemetryDrainRequested tracks the frozen legacy table, which no longer meters main sends, so it was never the right signal for this.

2 — reprojection depended on the marker. You're right that I described a sweep that does not exist. I did not add one; I changed the ordering and narrowed the claim to match.

The marker is now written before the projection is attempted rather than from the catch block. A marker written by an error handler cannot cover the case where the error handler never runs, which was exactly the window you found. As an intent record it survives a crash anywhere after it.

The residual gap is smaller but real, and the code comment and execution-evidence-spine.md now say so: a process that dies between the authority append and the marker write leaves a committed record Usage will not learn about. Closing that needs a full sweep over every run, which is a follow-up, not something I want to hide behind a comment.

4 — repair dropped the unreadable count. Confirmed. The call was real, and clearing the marker while discarding the count removed it from the totals and from the pending count in the same step. Clearing is still right — it will not decode on the next pass either, and holding the run would stall every later repair behind it — so the count now leaves on the result and folds into provenance.unreadableRecords next to stored rows that will not decode. Test covers it.

3 and 5 — left as follow-ups, and the PR description now says so instead of implying they are covered.

On 3: the renderer only ever called settings.usageStats(); the usage:* handlers had no consumer before this PR either. Giving Desktop the per-attempt retry/abort/provenance semantics means building the surface that displays them, which is #1625's job rather than something to bolt onto a metering change.

On 5: agreed, and the sharper half is that a later repair cannot update an already-archived review. Carrying provenance into the saved summary changes a persisted archive shape, so it wants its own change rather than riding along here.

CI is green. @maka/core 685/685, @maka/storage 950/951, @maka/runtime-host 488/488, @maka/runtime 2650/2662 (the remainder are Grep/filesystem tests that need rg as a binary, which my shell does not provide).

@Astro-HanAstro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the current head (8cbcd5d6). The previous blocking issue is resolved: the AgentRun stream is now the single durable authority, and the Usage ledger is a projection written only after the authority commits. The mixed-source hourly bucket and unpriced-row findings are also fixed with contract coverage.

I found no P0-P2 findings in the current revision. The remaining repair edge cases are P3 follow-ups:

  • a run-level marker can lose a wake-up under concurrent repair and projection failure;
  • an unreadable authority event is currently reflected only in the repair response that encounters it;
  • a permanently failing oldest repair batch can starve later markers.

These require compound failure conditions and do not block this PR. The implementation now matches the scope of #1679 and documents the residual authority-to-marker crash window honestly. CI is green. Approving.

@Astro-Han
Astro-Han merged commit ddeab1d into apache:mainAug 1, 2026
5 checks passed
@ARE404
ARE404 deleted the are404/feat-canonical-metering-seam branch August 1, 2026 19:05
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 2, 2026
First of the two compaction call kinds apache#1679 left unrouted. `history_compact`
was still writing a per-send row into the frozen `LlmCallRecord` table, which is
the last place a model call is metered outside `ModelCallAttempt`.
The summarizer already built a `ProviderRequestTracker` for capture and attempt
diagnostics; it now also carries accounting, so the call settles the same way a
main send does — one record per physical provider request, with `usageBasis` and
`costBasis` instead of a cost that cannot say whether it is real.
**Accounting is supplied per call, by the backend.** The host wires the
summarizer once at composition time and cannot know `runId`, which is per-turn
state only the backend holds. Rather than plumbing a turn-to-run resolver down
through the kernel and the backend factory context so the host could look up
something the backend already has, `AiSdkBackend.modelCallAccounting(callKind)`
builds the identity and the caller passes it in at the moment the call is made.
The main send path now uses the same factory, so the three call kinds share one
construction instead of repeating it.
The legacy writer goes out in the same commit: keeping both would have made
history compaction the double-metered path that apache#1755 removed everywhere else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First of the two compaction call kinds apache#1679 left unrouted. `history_compact`
was still writing a per-send row into the frozen `LlmCallRecord` table, which is
the last place a model call is metered outside `ModelCallAttempt`.
The summarizer already built a `ProviderRequestTracker` for capture and attempt
diagnostics; it now also carries accounting, so the call settles the same way a
main send does — one record per physical provider request, with `usageBasis` and
`costBasis` instead of a cost that cannot say whether it is real.
**Accounting is supplied per call, by the backend.** The host wires the
summarizer once at composition time and cannot know `runId`, which is per-turn
state only the backend holds. Rather than plumbing a turn-to-run resolver down
through the kernel and the backend factory context so the host could look up
something the backend already has, `AiSdkBackend.modelCallAccounting(callKind)`
builds the identity and the caller passes it in at the moment the call is made.
The main send path now uses the same factory, so the three call kinds share one
construction instead of repeating it.
The legacy writer goes out in the same commit: keeping both would have made
history compaction the double-metered path that apache#1755 removed everywhere else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Astro-Han pushed a commit that referenced this pull request Aug 3, 2026
…am (#1877)
* feat(metering): route history compaction through the canonical seam
First of the two compaction call kinds #1679 left unrouted. `history_compact`
was still writing a per-send row into the frozen `LlmCallRecord` table, which is
the last place a model call is metered outside `ModelCallAttempt`.
The summarizer already built a `ProviderRequestTracker` for capture and attempt
diagnostics; it now also carries accounting, so the call settles the same way a
main send does — one record per physical provider request, with `usageBasis` and
`costBasis` instead of a cost that cannot say whether it is real.
**Accounting is supplied per call, by the backend.** The host wires the
summarizer once at composition time and cannot know `runId`, which is per-turn
state only the backend holds. Rather than plumbing a turn-to-run resolver down
through the kernel and the backend factory context so the host could look up
something the backend already has, `AiSdkBackend.modelCallAccounting(callKind)`
builds the identity and the caller passes it in at the moment the call is made.
The main send path now uses the same factory, so the three call kinds share one
construction instead of repeating it.
The legacy writer goes out in the same commit: keeping both would have made
history compaction the double-metered path that #1755 removed everywhere else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(metering): route semantic compaction through the canonical seam
The second of the two compaction call kinds #1679 left unrouted, and the last
model call anywhere that was metered by hand. `semantic_compact` had no tracker
at all: it called `generateCompactSummary` directly and then built an
`LlmCallRecord` around the returned usage.
It now settles the way every other provider request does. `generateCompactSummary`
takes an optional `ProviderRequestTracker` and wraps the model with a
`wrapGenerate` middleware — the exact mirror of what `startStream` already does
with `wrapStream`, and in the one place that already owns "attach a tracker to a
model". The success, failure, and abort paths all settle inside the tracker, so
the hand-rolled try/catch that recorded three ways goes with it.
The backend hands the summarizer a *built* tracker rather than the capture,
attempt, and id sinks it is made of: compaction has no business assembling
metering identity, and a half-wired tracker is what produces records nothing can
attribute. `createProviderRequestTracker` also absorbed the main send's own
tracker construction, so this kind was added without a second copy of it.
One trace per turn rather than per call, so a step that summarizes is a step of
that trace and a retried summarization is another attempt of the same logical
call. Built on first use — most turns never summarize, and an unused trace id is
a trace that never happened.
The legacy writer goes out in the same commit, along with the now-unused
`computeCostUsd` dep: cost is resolved at settlement by the seam's own
`resolveCost`, with a basis attached.
One behavioural difference worth stating: the old row copied the SDK's
normalized `cacheRead` through as a cache hit. The canonical record attributes
cache tokens only when the provider's own payload claims them, so a provider
that reports none now yields an absent field instead of a number no provider
ever said.
`@maka/runtime` 2648/2661 (the 4 failures are the documented local `rg` noise).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(metering): drop the compaction path's legacy usage writer
With both compaction kinds routed, nothing in the send path writes a
`LlmCallRecord` any more. `AiSdkCompactionCapabilities.recordLlmCall` and its
`LlmTelemetryRecorder` type had no writer left above them, and the Host and
Desktop each still wired a recorder into a backend field nothing read — dead
plumbing that a future change could just as easily have brought back to life.
Five backend tests still collected the records into arrays nothing asserted on;
those go too.
**`recordLlmCall` itself stays, and #1679's plan to delete it here was wrong.**
`goal_evaluation` is the fourth call kind, and `createHostGoalEvaluator` still
writes it to the frozen table. It cannot follow the compaction kinds through
this seam as-is: a `ModelCallAttempt` is identified by `(sessionId, runId,
turnId)`, and the Host evaluates a goal against a `sessionId` alone — there is
no run or turn at that layer to attribute the call to. Giving it one is a design
question for the RFC, not something to smuggle into a routing change.
The practical consequence is worth stating plainly, because #1679 currently says
otherwise: `provenance.legacyRecords` will *not* fall to zero as the old table
ages out. Goal evaluations keep landing there, so the merged read path stays
load-bearing until that kind is routed too.
`@maka/runtime` 2648/2661 (4 = documented local `rg` noise), `@maka/runtime-host`
488/488, `@maka/desktop` 1090/1128 — identical to this worktree's baseline on the
parent commit (the 38 are its missing Astryx peer dep, not this change).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(metering): stop gating metering on the capture sink, and pin what it settles
Review round 1 on #1877. The P1 and the P2s that were about this PR's own
behaviour, plus every P3.
**Metering no longer depends on a diagnostic.** `createProviderRequestTracker`
returned undefined without `recordProviderRequestCapture`, so a deployment with
capture off — a reachable config; both hosts wire it conditionally — silently
stopped metering compaction, which the deleted `recordLlmCall` had metered
unconditionally. Capture is now optional inside the tracker: `preparedCapture`
is pure, so `requestHash`, `requestBytes`, and `segments` survive without a
sink, and only the artifact join keys (`captureId`, `captureArtifactId`) go
absent. The tracker is built when there is either sink to feed. The Host now
wires the summarizer's tracking unconditionally too, and the Desktop wires it at
all — it never had, so the accounting the backend computed for a Desktop history
compaction was handed to a summarizer that had nowhere to settle it.
**The "one place owns attaching a tracker" claim is now true rather than
asserted.** `withProviderGenerateTracking` is shared by `generateCompactSummary`
and `buildLlmHistorySummarizer`; `ProviderMiddlewareGenerateInput` is declared
once. Also fixed: history_compact records carried no `contextWindow` while
semantic_compact did.
**Tests for what was only claimed:**
- `usageBasis: 'missing'` — the branch had no test anywhere. A call the provider
reported no usage for records `missing`, not zero tokens, and stays unpriced
whatever the resolver would have said.
- Metering with capture switched off, asserting the attempt still carries the
locally-computed request shape and no artifact ids.
- Cache attribution end-to-end: the semantic mock now ships a `raw` provider
payload, so the assertion moved from "absent because this mock claims nothing"
to `cacheReadInputTokens: 2` through the provider branch. The no-`raw` rule
stays pinned at the unit level.
- A dry-run (`validate_only`) semantic compaction really is a billed call: the
summarizer runs to completion and only then is its block refused. Confirmed by
the test, which is why it is worth pinning — a mode named "dry run" that bills
is what a later reader would assume otherwise.
- Mid-turn history compaction settles a canonical record end-to-end through the
backend glue, with the real summarizer against a mock provider, asserting the
live `runId` resolves — a stubbed resolver cannot show that.
P3 cleanups: two dead imports, and the Usage IPC comment that still claimed the
frozen table receives compaction calls.
`@maka/runtime` 2684/2696 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 524/524, `@maka/desktop` 1101/1139 (38 = this worktree's
missing Astryx peer dep, unchanged from the parent commit).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(headless): narrow the trace analyzer to optional capture ids
Fallout from making capture optional on `ProviderRequestAttemptRecord`: the
headless provider-request trace analyzer joins attempts to captures by id, and
the id is no longer guaranteed to be there.
Behaviour is unchanged for every trace this tool actually reads. It analyses a
capture ledger, so an attempt that cannot name a capture is incomplete by its
own definition and fails with the same diagnostic it already produced — and the
decoder above it already rejects such records as `invalid_attempt` before the
join runs. Only the type needed narrowing.
Caught by CI, not by me: I checked the blast radius of the optional fields in
`core`, `storage`, `runtime-host`, and `desktop`, and did not grep `headless`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(metering): name the run for a manual compaction, and price each call as its own model
Review round 2 on #1877. Both findings were correct, and the first was a live
bug I had argued my way out of.
**Manual compaction was unmetered, and my "nothing reaches this path" was
wrong.** Desktop `sessions:compact` and CLI `/compact` both reach
`RuntimeKernel.compactSession()`, which opens an `AgentRun` and then called
`compactHistory` without its id. Outside `send()` there is no `currentRunId`, so
the summarization settled nothing while the old `recordLlmCall` had metered it.
`BackendCompactHistoryInput.runId` is now **required**, not optional: the failure
mode is silence, and an optional field is one a caller can forget in exactly the
situation that produced this bug. The kernel passes `run.runId`; the compaction
threads it into the accounting identity for that one call. Seventeen test call
sites had to name a run, which is the type doing its job.
**A configured summarizer model was priced as the session model.** Cost
resolution looked up `${providerType}:${this.input.modelId}` regardless of which
model served the request, so with `MAKA_CONTEXT_SEMANTIC_COMPACT_MODEL` set we
stored one model's id beside another model's `pricingRates` — precisely what
recording the rates exists to prevent. `resolveModelCallCost` now takes the
call's model id, supplied through the same identity the tracker already carries.
**One bug the new test caught in my own fix:** the compaction dep was wired as
`(callKind) => this.modelCallAccounting(callKind)`, which silently dropped the
new identity argument. TypeScript accepts a narrower function, so the manual
compaction kept recording nothing and the types stayed green. Only the
end-to-end assertion showed it.
Tests, both as asked: `SessionManager.compactSession()` settles exactly one
`history_compact` record carrying the run the kernel opened, driven through a
real `AiSdkBackend` and a real summarizer; and a semantic compaction with a
distinct summarizer model records that model's own rates while the send's own
steps keep the session model's.
`@maka/runtime` 2686/2698 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 524/524, `@maka/desktop` 1101/1139 (38 = this worktree's
missing Astryx peer dep). All runtime-consuming packages build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor(metering): give the backend sole ownership of the compaction tracker
Review round 3 on #1877. The blocker was real: the CLI composition never passed
`ctx.recordModelCallAttempt` to `AiSdkBackend` and never configured the
summarizer's tracking, so `/compact` got the new `runId` and still had nothing to
settle into. Harbor had the same gap for `semantic_compact`. Both are fixed here,
but the reviewer's structural point is the one worth acting on: this is the fifth
way the same split has failed, so the split goes.
`history_compact` now gets its tracker from the same
`AiSdkBackend.createProviderRequestTracker` that `main` and `semantic_compact`
already use, and `HistoryCompactSummaryInput` carries a *ready* tracker instead
of an accounting identity. `BuildLlmHistorySummarizerOptions.providerRequestTracking`
is gone, and with it the Host's and Desktop's hand-assembled copies of the
tracker's inputs — including the `contextWindow` plumbing added one round ago,
which the backend now supplies for free because it knows the model.
What a product owes accounting is now exactly one thing: the canonical sink.
Nothing else can be half-wired, because nothing else is a product's to wire.
`AiSdkCompactionDeps.modelCallAccounting` went with it — one factory, one seam.
**Composition tests, because the last round's test proved the wrong thing.** The
`SessionManager.compactSession()` test supplied both dependencies itself, so it
demonstrated that the runtime works when everything is wired, not that the CLI
wires it. The two new tests build backends through the real CLI and Harbor
factories and assert the sink reaches the caller's recorder. Both were confirmed
to fail with their production wiring reverted.
CLI's gap turned out to be wider than the review described: with no sink passed
at all, `main` sends were unmetered too, and had been since the frozen table's
writer was removed — CLI never wired `recordLlmCall` either, so this composition
root has simply never been inside accounting.
Left out deliberately, as agreed: `goal_evaluation` routing, a global
`errorClass`, and anything Inspector-shaped.
`@maka/runtime` 2700/2712 (3 suites = documented local `rg` noise),
`@maka/runtime-host` 563/563, `maka-agent` (CLI) 467/467, `@maka/headless`
harbor-cell 102/102, `@maka/desktop` 1156/1194 (38 = this worktree's missing
Astryx peer dep).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First slice of apache#1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.
Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.
Three properties are deliberate:
- **Retries are nested, not flattened.** Attempts of one logical call share a
`logicalCallId` by contract, so "this call was retried twice" is a grouping
rather than something a reader reconstructs from four steps that happen to
share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
no `costUsd`, and a session of only such calls totals to no price rather than
to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
produces a trace that says so. The pi backend is exactly this case: it emits
`token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
indistinguishable from a session that did nothing. Aggregate usage with no
record behind it is the signal, reported per session as `absent` or `partial`
with the turns named.
Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.
No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.
`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ARE404 added a commit to ARE404/maka-agent that referenced this pull request Aug 3, 2026
First slice of apache#1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (apache#1687, apache#1755, apache#1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.
Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.
Three properties are deliberate:
- **Retries are nested, not flattened.** Attempts of one logical call share a
`logicalCallId` by contract, so "this call was retried twice" is a grouping
rather than something a reader reconstructs from four steps that happen to
share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
no `costUsd`, and a session of only such calls totals to no price rather than
to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
produces a trace that says so. The pi backend is exactly this case: it emits
`token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
indistinguishable from a session that did nothing. Aggregate usage with no
record behind it is the signal, reported per session as `absent` or `partial`
with the turns named.
Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.
No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.
`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Astro-Han pushed a commit that referenced this pull request Aug 3, 2026
…#1956)
* feat(inspector): project a per-session causal trace over both ledgers
First slice of #1625, and the one that was blocked: the RFC's headline —
per-step latency and cost — was not derivable from `RuntimeEvent`, which is why
this issue paused for the accounting work (#1687, #1755, #1877). That work
landed, so `ModelCallAttempt` now carries one record per physical provider
request with `step`, `attempt`, latency, time-to-first-token, usage basis, and a
cost frozen at call time. The projection this issue always wanted is now
writable honestly.
Pure and synchronous: both ledgers are handed in already read, so the caller
owns the I/O and `@maka/storage` stays out of `@maka/runtime`. Contract in
`packages/core`, builder in `packages/runtime`, per the split agreed on the
issue.
Three properties are deliberate:
- **Retries are nested, not flattened.** Attempts of one logical call share a
`logicalCallId` by contract, so "this call was retried twice" is a grouping
rather than something a reader reconstructs from four steps that happen to
share an id.
- **An absent price stays absent.** A step whose attempts were never priced has
no `costUsd`, and a session of only such calls totals to no price rather than
to zero — the distinction the canonical record exists to keep.
- **Coverage is stated, not implied.** A backend that emits no canonical records
produces a trace that says so. The pi backend is exactly this case: it emits
`token_usage` and zero `ModelCallAttempt`, and an empty timeline would be
indistinguishable from a session that did nothing. Aggregate usage with no
record behind it is the signal, reported per session as `absent` or `partial`
with the turns named.
Failure attribution points at what failed *first*, not at the terminal error: a
turn that ends in an error usually ends there because of an earlier tool
failure, and naming the last event names the symptom.
No UI, no search API, no cost authority of its own — a per-session total sums
the same `ModelCallAttempt` records Settings → Usage aggregates, read at a
different scope.
`@maka/core` 703/703, `@maka/runtime` 2721/2733 (3 suites = documented local
`rg` noise).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(inspector): correct six attribution and coverage gaps in the trace
Review on #1956. All six were right; four are cases the tests never drove and
two were contract surface that promised more than the code delivered.
**Coverage claimed a proof it cannot have.** `complete` said every call settled;
the canonical contract says present settlements cannot prove completeness. The
state is now `no_known_gap` — the absence of evidence of a gap. A detectable
shortfall is also new: `tokenUsage.runtimeSteps` states how many tool-loop steps
one aggregate stands for, and fewer main calls on record than that is a
disagreement between the two ledgers, reported per turn as a floor on what is
missing rather than a count of it.
**Attempts are deduplicated by `attemptId`.** An aborted attempt and its later
settlement are appended under one id; the ledger dedupes on write, a stream read
does not. Without `dedupeModelCallAttempts` the trace invented a retry and
double-counted a priced settlement, which would have put a session total out of
step with Settings → Usage over the very same records. Grouping now goes through
core's `groupModelCallAttempts`, which dedupes on the way in.
**Step-less turns had non-finite bounds.** Usage-only and text-only turns project
no steps, and folding an empty list gives ±Infinity — which JSON renders as
`null`. Bounds now come from the ledger facts the turn is made of.
**A handled tool failure no longer fails the turn.** Any failed step marked the
whole turn failed, including a tool error the model recovered from before
finishing normally. The ledger's terminal status decides whether the turn failed;
the failed step only locates the cause once that is established.
**The compaction step is emitted rather than merely declared.** `TraceCompactionStep`
existed and nothing produced it. Written checkpoints are system text events, and
they are a different fact from the `history_compact` model call: one is the
boundary the next request replays from, the other is the spend. Both now appear.
**`recoveryMode` was a policy wearing the name of an outcome.** Every dispatch
declares one, including ordinary first executions. It is now `recoveryPolicy`,
and an actual `recovered` decision is joined from `actions.toolRecovery` by
`operationId` — correlated rather than positional, because the decision is
appended by the recovery writer and not by the dispatch it settles.
Seven regression tests, one per finding plus the no-known-gap state.
`@maka/core` 703/703, `@maka/runtime` 2742/2757 — the 6 local failures are this
machine's `rg`-as-a-shell-function and macOS path noise; CI ran the same base
green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Astro-Han added a commit that referenced this pull request Aug 3, 2026
Opening the read path made an existing conversation-copy gap reachable.
`isCopiedAgentRunEvent` is a blocklist, so an event type this build does not
know passed it, and `cloneAgentRunEvent` rewrites payload references only for
the three schemas it understands — everything else kept its source session,
run, and artifact ids while the envelope was rewritten onto the target. Before
the decoder opened up this was a loud failure (strict read threw, or the record
became `event_corrupt`, which the cloner rejects outright); it would now be a
silent one. Drop those events instead, which is what this function already does
for terminal and compact-block events it cannot safely transform.
Restore the write-side protection at the layer that owns it. Narrowing
`AgentRunStore.appendEvent` was the wrong seam — that port must still accept
historical events being replayed. Producers are the right seam:
`terminalAgentRunEventType` returns `AgentRunEventType` again rather than the
now-open `AgentRunEvent['type']`, and `RunTraceEventType` carries a compile-time
proof that it stays a subset of the emitted catalogue, so the two lists #1755
had to edit in lockstep can no longer drift apart silently.
Use a plain `string` for the persisted read type. `AgentRunEventType | (string &
{})` bought literal completion at the cost of blocking `===` narrowing at every
consumer; `string` still cannot be assigned to `AgentRunEventType`, so the
narrow projection-key parameters stay protected. That removes the headless
narrowing helper entirely and both SQLite substitutions; only the file store's
async projection callback still needs the literal constant, because TypeScript
drops parameter narrowing inside a closure.
Cover the paths that actually failed: cutover into the SQLite store the desktop
app uses, and the empty, non-string, and extra-field records that the envelope
check is now the only thing rejecting.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
…ntract
Opening `AgentRunEvent.type` to any string let the read tolerance leak into
writes: every producer builds a bare object literal, so a typo compiled, both
stores persisted it, and every consumer — matching on exact types — ignored it.
Deleting a type still in use was equally silent; removing `task_gate_decided`
from the catalogue type-checked cleanly across four packages even though two
call sites still write it. That is the #1755 failure mode, unguarded.
`EmittedAgentRunEvent` narrows `type` back to `AgentRunEventType` and
`appendEvent` takes it, so a misspelled or retired type now fails at the append
that would persist it. Reads stay on the open `AgentRunEvent`. This became
available once the previous commit stopped conversation copy from carrying
unknown events: nothing writes an untyped event any more, so the write port no
longer has to accept one.
Narrowing the port made three of this branch's own additions redundant, all now
gone: the `RunTraceEventType ⊆ AgentRunEventType` assertion (`traceToRunEvent`
proves it at the conversion), the `CHECKPOINT_PROJECTION_TYPE` constant and both
substitutions (`event.type` narrows again), and the emitted-catalogue array cast
in the copy filter (now a type guard). `run-trace.ts` returns to its `main` state.
Two remaining gaps from the same open-string change. Harbor's trace validator
reads artifacts written by one build, so version tolerance is wrong there — an
unrecognized type is corruption and is reported again. And strict decode
accepted whitespace discriminants like `" "`, which no conforming writer emits.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
An AgentRun ledger is append-only and outlives the build that wrote it, but
`decodeAgentRunEvent` rejected any event whose type was absent from
`AGENT_RUN_EVENT_TYPES` — conflating "this build does not write it" with "this
record is damaged". #1755 retired the `usage_recorded` writer and dropped the
enum entry in one step, which is correct for the writer and retroactively
illegal for every ledger already on disk: v0.1.3 then refused to start on any
machine that had run v0.1.2 (#1942).
Decode now accepts any non-empty string type and keeps validating the envelope,
so strict recovery still rejects genuinely damaged records while tolerating both
retired types and types from a newer build a user downgraded away from — a
direction no enum discipline can cover. `AGENT_RUN_EVENT_TYPES` becomes what
this build emits rather than what it can read, and the `usage_recorded` entry
added by #1945 is dropped as the dead entry it now is.
Fixes#1942
Astro-Han added a commit that referenced this pull request Aug 3, 2026
Opening the read path made an existing conversation-copy gap reachable.
`isCopiedAgentRunEvent` is a blocklist, so an event type this build does not
know passed it, and `cloneAgentRunEvent` rewrites payload references only for
the three schemas it understands — everything else kept its source session,
run, and artifact ids while the envelope was rewritten onto the target. Before
the decoder opened up this was a loud failure (strict read threw, or the record
became `event_corrupt`, which the cloner rejects outright); it would now be a
silent one. Drop those events instead, which is what this function already does
for terminal and compact-block events it cannot safely transform.
Restore the write-side protection at the layer that owns it. Narrowing
`AgentRunStore.appendEvent` was the wrong seam — that port must still accept
historical events being replayed. Producers are the right seam:
`terminalAgentRunEventType` returns `AgentRunEventType` again rather than the
now-open `AgentRunEvent['type']`, and `RunTraceEventType` carries a compile-time
proof that it stays a subset of the emitted catalogue, so the two lists #1755
had to edit in lockstep can no longer drift apart silently.
Use a plain `string` for the persisted read type. `AgentRunEventType | (string &
{})` bought literal completion at the cost of blocking `===` narrowing at every
consumer; `string` still cannot be assigned to `AgentRunEventType`, so the
narrow projection-key parameters stay protected. That removes the headless
narrowing helper entirely and both SQLite substitutions; only the file store's
async projection callback still needs the literal constant, because TypeScript
drops parameter narrowing inside a closure.
Cover the paths that actually failed: cutover into the SQLite store the desktop
app uses, and the empty, non-string, and extra-field records that the envelope
check is now the only thing rejecting.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
…ntract
Opening `AgentRunEvent.type` to any string let the read tolerance leak into
writes: every producer builds a bare object literal, so a typo compiled, both
stores persisted it, and every consumer — matching on exact types — ignored it.
Deleting a type still in use was equally silent; removing `task_gate_decided`
from the catalogue type-checked cleanly across four packages even though two
call sites still write it. That is the #1755 failure mode, unguarded.
`EmittedAgentRunEvent` narrows `type` back to `AgentRunEventType` and
`appendEvent` takes it, so a misspelled or retired type now fails at the append
that would persist it. Reads stay on the open `AgentRunEvent`. This became
available once the previous commit stopped conversation copy from carrying
unknown events: nothing writes an untyped event any more, so the write port no
longer has to accept one.
Narrowing the port made three of this branch's own additions redundant, all now
gone: the `RunTraceEventType ⊆ AgentRunEventType` assertion (`traceToRunEvent`
proves it at the conversion), the `CHECKPOINT_PROJECTION_TYPE` constant and both
substitutions (`event.type` narrows again), and the emitted-catalogue array cast
in the copy filter (now a type guard). `run-trace.ts` returns to its `main` state.
Two remaining gaps from the same open-string change. Harbor's trace validator
reads artifacts written by one build, so version tolerance is wrong there — an
unrecognized type is corruption and is reported again. And strict decode
accepted whitespace discriminants like `" "`, which no conforming writer emits.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
`AGENT_RUN_EVENT_TYPES` was both the set this build writes and the set a reader accepts.
The ledger is append-only and outlives the build that wrote it, so retiring a writer and
deleting its entry in one commit (#1755) left the next build unable to decode records the
previous one had persisted, and the desktop app failed to start (#1942).
Only the write side can be a closed set. Reads take `type` as an open string with the
envelope around it still validated, so a damaged record is still rejected; appends take
`EmittedAgentRunEvent`, so a misspelled or retired type fails to compile at the call that
would persist it. That makes retiring a writer safe, which is why `usage_recorded` is gone
again: #1945 could only fix the crash by adding a type nothing writes back to the catalogue
this build writes from.
A copy drops an event this build does not emit rather than carrying its unrewritten
source-owned ids into the target, since the rewriters cannot inspect a payload they do not
know.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
`AGENT_RUN_EVENT_TYPES` was both the set this build writes and the set a reader accepts.
The ledger is append-only and outlives the build that wrote it, so retiring a writer and
deleting its entry in one commit (#1755) left the next build unable to decode records the
previous one had persisted, and the desktop app failed to start (#1942).
Only the write side can be a closed set. Reads take `type` as an open string with the
envelope around it still validated, so a damaged record is still rejected; appends take
`EmittedAgentRunEvent`, so a misspelled or retired type fails to compile at the call that
would persist it. That makes retiring a writer safe, which is why `usage_recorded` is gone
again: #1945 could only fix the crash by adding a type nothing writes back to the catalogue
this build writes from.
A copy drops an event this build does not emit rather than carrying its unrewritten
source-owned ids into the target, since the rewriters cannot inspect a payload they do not
know.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
`AGENT_RUN_EVENT_TYPES` was both the set this build writes and the set a reader accepts.
The ledger is append-only and outlives the build that wrote it, so retiring a writer and
deleting its entry in one commit (#1755) left the next build unable to decode records the
previous one had persisted, and the desktop app failed to start (#1942).
Only the write side can be a closed set. Reads take `type` as an open string with the
envelope around it still validated, so a damaged record is still rejected; appends take
`EmittedAgentRunEvent`, so a misspelled or retired type fails to compile at the call that
would persist it. That makes retiring a writer safe, which is why `usage_recorded` is gone
again: #1945 could only fix the crash by adding a type nothing writes back to the catalogue
this build writes from.
A copy drops an event this build does not emit rather than carrying its unrewritten
source-owned ids into the target, since the rewriters cannot inspect a payload they do not
know.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@ARE404@Astro-Han