fix(runtime): RuntimeEvent ledger is recovery-grade (WAL), not source-of-truth — write-path truncation blocks counterfactual replay

[Astro-Han]

Goal vs reality

The RuntimeEvent ledger is designed to be the source of truth for a run: any view (prompt, projection, evidence export) should be rebuildable from it — including counterfactual views under different context policies, which is the foundation for replay-based policy evaluation (free pre-filtering of context-budget candidates for the RSI loop) and for AHE evidence fidelity.

An audit (2026-07-07) shows the current implementation is recovery-grade (WAL): it guarantees resuming the view the model actually saw, but cannot rebuild a different view (write path loses pre-policy originals) nor byte-identically rebuild the same view (read path depends on live state that is never recorded).

Two criteria define source-of-truth grade:

1. Facts are written before policy — no truncation/cleaning/budgeting on the write path; all "show the model less" transforms are read-time projections.
2. Read is a pure function of (ledger, policy params) — no live clock/fs/config/registry inputs, or their values are snapshotted per run.

The read side already follows criterion 1's spirit everywhere (all prune/compact mechanisms are projection-time, SHA-verified archives, refuse-to-substitute on archive failure). The violations are at the source and at the edges.

Write-path violations (criterion 1)

• F1 (worst): Bash output truncated to 2000 lines / 50 KB before persistence. shapeTerminalResult calls truncateToolOutput inside the tool impl (packages/runtime/src/shell-tools.ts:148-157, also builtin-tools.ts:206; limits in tool-output.ts:48-49), so the ledger's canonical record is the truncated view. The header comment (tool-output.ts:12-14) documents this as a deliberate choice for the isolated-executor benchmark path. Local path is partially recoverable via ShellRunStore 1 MB tail (shell-run-manager.ts:353-389, linked by sourceToolCallId); the harbor isolated-executor path has no full-output record at all.
• F2: capture-time 1 MB cap — BASH_MAX_RETAINED_CHARS (shell-exec.ts:27); BashTailBuffer.trim() (bash-tail-buffer.ts:58-83) discards beyond-cap bytes as they stream. The head of a >1 MB output never exists on disk.
• F3: Pi backend caps tool-result text at 8192 chars before the ToolResultEvent is created (pi-agent-backend.ts:543-553).
• F4 (minor): synthetic tool error text sliced to 4000 chars before persist (tool-runtime.ts:87, 735-740).
• F5 (accepted exception, keep): secret redaction is write-time (shell-tools.ts:146-147 etc.). Correct as-is — secrets must never touch disk. Document as the one sanctioned deviation from criterion 1.

Note the downstream irony: the archive machinery (SHA-256, verified read-back) faithfully preserves content that was already lossy when it reached the ledger.

Read-path impurities (criterion 2)

The projection core is verified pure (applyRuntimeEventContextBudget / applyRuntimeEventHistoryCompact, context-budget.ts:457,566; tool re-seeding tool-availability.ts:198-223). Purity breaks at the edges:

• Turn tail unrecorded: live date/platform (system-prompt/session-environment-prompt.ts:19-20), git branch (system-prompt/project-context.ts:22), live background-shell summary (ai-sdk-backend.ts:789-792, 2347-2349) are appended to the user message at send time; the ledger stores only userInput.text.
• System prompt live-read each turn: instruction files + settings (system-prompt/workspace-instructions.ts:139, cli-system-prompt.ts:20-44); only systemPromptHash is persisted (request-shape.ts:94) — drift is detectable but not reproducible.
• Policy knobs mostly unrecorded: ~50 MAKA_CONTEXT_* env vars gate sub-policies (context-budget-policy.ts:15-16 ff.); the persisted ContextBudgetDiagnostic records names/flags but not ratios, minRecentTurns, per-block caps.
• Tool schema bodies unrecorded: only toolSchemaHash (request-shape.ts:96-103).

Impact

• Counterfactual replay is one-directional today: simulating tighter policies over recorded ledgers is faithful; looser (e.g. "what if the Bash cap were 100 KB") is impossible — the data never existed.
• Byte-exact replay of even the same view is impossible; only hash-level drift detection works.
• Historical runs can never be upgraded retroactively; every day without the capture fix is golden-fixture data not being accumulated.

Fix plan (priority order)

1. Move F1/F3 truncation to read-time projection: persist the full (redacted) tool output as an artifact (reuse the existing SHA-256 archive machinery — harbor-cell.ts:916-959 proves it exists on the isolated path too), keep a ref in the ledger, make the 50 KB/8 KB budgets projection parameters.
2. Emit a request_captured event at send time (ai-sdk-backend.ts:857 area): canonical JSON of resolved system prompt, turn tail, effective ContextBudgetPolicy snapshot, tool schema bodies (content-addressed). One change closes all four read-path holes and provides ground truth for the replay test.
3. Keep the 1 MB capture cap (F2) but record dropped-byte count so replay knows the uncertainty bound.
4. Fix F4 alongside 1.

Replay acceptance test (can start now)

1. Hash-level (no runtime change): replay a maka-eval fixture ledger through AiSdkBackend + MockLanguageModelV3 (existing harness pattern, __tests__/ai-sdk-backend.test.ts asserts on doStreamCalls[n].prompt), recompute computeRequestShapeDiagnostic, assert systemPromptHash/toolSchemaHash/requestShapeHash equal the recorded model_stream_started values.
2. Content-level once item 2 lands: byte-compare re-assembled {system, messages, tools} against the captured event. New runs only.
3. Wire-level: inject a recording fetch via model-factory.ts:19,56,64, diff literal HTTP bodies.

Acceptance criteria

• A recorded run replays offline with byte-identical {system, messages, tools} (post item 2, new runs)
• Full tool outputs recoverable from artifacts regardless of model-facing budget (post item 1)
• Contract guarded by CI: any future write-path truncation or unrecorded live input turns the replay test red

[Astro-Han]

Independent verification pass (Codex, read-only, per-claim against source). Central thesis CONFIRMED: write path has pre-ledger loss (F1/F2/F4 confirmed at cited lines; ledger append path verified through ai-sdk-flow.ts:227 → agent-run.ts:441), read path depends on unsnapshotted live inputs, projection core purity confirmed, read-time prune/compact mechanisms confirmed non-mutating with SHA-verified archives.

Five statements need narrowing before this is used as a fix spec:

1. F3 is too broad. Only bare-string Pi tool results get the 8192-char cap (pi-agent-backend.ts:543-553); results already shaped as {kind: ...} objects go through redactUnknown() with no per-field cap (:544,556).
2. toolSchemaHash is NOT persisted in model_stream_started. The trace event carries systemPromptHash and requestShapeHash only (run-trace.ts:104, agent-run.ts:928); toolSchemaHash is computed inside the request-shape diagnostic but never emitted. Replay test step 1 must drop that assertion or the capture must be added first.
3. History projection hash is not uniformly shape-only. Plain text is hashed as role + char count, but tool-result output payloads DO get a content hash via stableHash(value) (request-shape.ts:282-318). Tool outputs are therefore content-verifiable today; only message text is not.
4. Injectable fetch only reaches the claude-subscription/codex-subscription branches (model-factory.ts:15,52,71); plain openai/google/openai-compatible providers don't receive it. Byte-diff test step 3 requires wiring fetch through the remaining provider branches first.
5. "System prompt live-read each turn" is not uniform. resolveSystemPrompt() is invoked per send, but the desktop local-memory fragment is snapshotted at backend creation (apps/desktop/src/main/main.ts:631). Conclusion unchanged (resolved prompt body is still unrecorded), mechanism corrected.

One adjacent write-path finding the audit missed: SpawnChildAgent parent-side tool result stores at most a 4000-char summary (session-manager.ts:532,1032). Softer than F1/F3 since the full child run persists in the child's own ledger (recoverable via AgentOutput), but strictly the parent run's ledger is not self-sufficient.

[Astro-Han]

@likun666661 This audit touches the ledger/evidence surfaces you own (AHE target protocol #411, evidence export #413) — please review both the findings and the verification corrections above. Two points most relevant to AHE: (1) F1/F3 mean trace-index.json evidence currently references pre-truncated tool outputs, so AHE's failure evidence is lossy at the source; (2) the fix plan's request_captured event would give the evidence export a content-exact request record instead of hash-only. Push back if any finding mischaracterizes the design intent.

[likun666661]

补充 review 结论：我按当前 upstream/main@a6d99e0d 过了一遍，#615 的核心判断是成立的。

这不是普通的 resume/runtime correctness bug，而是 RuntimeEvent 当前更像 recovery-grade WAL：它足够恢复“当时已经投给模型的 lossy view”，但还不能作为 AHE / RSI / counterfactual replay 的 source of truth。关键原因有两类：一类是写入 RuntimeEvent 之前已经把事实截断/投影掉；另一类是 provider request 的若干输入仍在发送时 live resolve，只记录 hash/诊断，不记录 byte-exact request body。

受影响模块主要是这几组：

1. Shell/tool 输出捕获与投影：packages/runtime/src/shell-exec.ts, bash-tail-buffer.ts, shell-run-manager.ts, shell-tools.ts, builtin-tools.ts, tool-output.ts。
2. Tool result 到 RuntimeEvent 的落盘链路：tool-runtime.ts, ai-sdk-flow.ts, agent-run.ts, runtimeEventStore, session-manager.ts。
3. Provider request 构造与 request shape：ai-sdk-backend.ts, model-adapter.ts / model-factory.ts, request-shape.ts。
4. Live prompt/context 输入：system-prompt/session-environment-prompt.ts, project-context.ts, workspace-instructions.ts, shell run context summary / turn tail prompt。
5. 后续 projection/archive/evidence surface：context-budget.ts, active-tool-result-prune.ts, semantic-compact.ts, packages/headless/src/harbor-cell.ts, desktop tool-result archive artifacts, result-export.ts, AHE docs。

一个具体的 write-path 丢失序列是：

1. 用户启动 run，AgentRun.begin() 只把 user text / attachments 这一层记录下来。
2. LLM 调 Bash，Bash 产生很长输出，例如 5MB。
3. shell-exec.ts / BashTailBuffer 只保留约 1MB tail，head 已经丢掉。
4. shapeTerminalResult() 再通过 truncateToolOutput(..., { direction: 'tail' }) 投影到模型可见预算，默认约 2000 lines / 50KB。
5. ToolRuntime 发出的是已经 shaped 过的 tool_result session event。
6. ai-sdk-flow.ts 把它映射成 RuntimeEvent 的 function_response.result。
7. AgentRun.recordRuntimeEvents() 再把这个 RuntimeEvent 落到 ledger。
8. 后面的 replay / archive / AHE evidence 只能看到这份已经截断后的结果。想用更宽松 policy 做 counterfactual replay 时，原始 head bytes 已经不存在。

Pi/tool-result 这边也有类似边界，不过需要说精确一点：Pi backend 对 bare string result 会走 redactBoundedText()，默认 8192 chars；object-shaped result 走的是 redactUnknown()，不是同一个 8192 cap。synthetic tool error 还有 TOOL_ERROR_RESULT_MAX_CHARS = 4000 的单独 cap。

一个具体的 read-path 非纯序列是：

1. t0 发送模型请求时，ai-sdk-backend.ts live resolve system prompt、turn tail prompt、shell run summary、workspace instructions、git/project context、tool schema、context budget policy 等。
2. provider 实际收到的是这些 live bytes 组合出的 { system, messages, tools, providerOptions... }。
3. 当前 trace/ledger 记录了 user text、部分 prompt/request shape hash 和诊断，但没有记录完整 canonical request body，也没有把 tool schema / effective context policy 作为可重建 snapshot 落下来。
4. t1 用同一 RuntimeEvent ledger replay，如果日期、git branch、workspace instructions、tool registry 或 env policy 变了，要么 rebuild 出不同 provider bytes，要么只能靠 hash 发现 drift，却不能 byte-exact reconstruct 原请求。

几个边界/修正也建议写进方案里：

• request-shape.ts 里确实计算了 toolSchemaHash，但 model_stream_started 目前只持久化 systemPromptHash / requestShapeHash 等字段，没有把独立 toolSchemaHash 作为事件字段发出来。
• historyProjectionHash 对 tool-result output payload 会做 content-ish stable hash；普通 text 主要是 shape/length 级别，不是完整文本 hash。
• injectable fetch 当前不是所有 provider branch 都贯通，wire-level replay test 需要先把 provider wiring 补齐。
• write-time secret redaction 仍然应该保留；secret 不应为了 replay 以 raw 形式持久化。这里需要的是“full redacted artifact before model-facing budget”，不是 raw secret ledger。
• SpawnChildAgent 父 run 里的工具结果 summary 是 lossy 的，但 child ledger 本身还可以另查；这点不要和父 tool result 的 lossy summary 混成同一个不可恢复面。

所以我会把修复方向拆成两层：

1. Tool/output source-of-truth：在任何 model-facing budget/truncation 之前，先持久化 full redacted output artifact，RuntimeEvent 里记录 content-addressed ref、size/hash、redaction metadata、projection policy。50KB/8KB/4000-char 这些应该是“给模型看的 projection 参数”，不是 source-of-truth 的存储上限。如果底层捕获仍然只有 1MB tail，也要显式记录 dropped-byte / uncertainty metadata。
2. Provider request snapshot：在 provider send 边界新增类似 request_captured 的事件，记录 canonical {system,messages,tools,providerOptions,effectiveContextBudgetPolicy}。大体积字段可以 artifact 化，但 replay 必须能从 ledger/artifact 还原 byte-exact request，不能再依赖 live workspace/git/env/tool registry。

测试也建议分层做：先做 hash-level regression，确认当前 hash drift 能被稳定检测；有 request_captured 后做 content-level replay，确认从 ledger/artifact 重建出的 request bytes 和原请求一致；等 provider fetch 注入贯通所有分支后，再补 wire-level capture/replay 测试。