Tui UI

CLAUDE.md

@@ -37,7 +37,7 @@ The v4 architecture systems are implemented and live in `apps/core/src/`:
 | **Hooks** | `hooks/` — PreToolUse, PostToolUse, PostToolUseFailure, PermissionRequest, PreCompact, PostCompact, SessionStart, UserPromptSubmit, SubagentStart, SubagentStop, Stop, Notification, TurnStart, TurnEnd |
 | **Skills System** | `skills/manager.ts`, `skills/loader.ts`, `skills/registry.ts`, `skills/injection.ts`, `skills/types.ts` |
 | **Rollout/Event Sourcing** | `rollout/recorder.ts`, `rollout/types.ts`, `rollout/history.ts`, `rollout/replay.ts` |
-| **Observability** | Spec `2026-08-10-agent-observability.md`. Model-call events (`model.request`/`first_token`/`response`/`error`) recorded by `rollout/recorder.ts` around `callProviderOnce`; `model.request` is written *before* the call so an unterminated request is itself the evidence of a hang. **Timeouts live at the fetch layer** (`providers/fetch-timeout.ts`, wired as the `fetch` option on every provider): 300s for response headers, 180s of silence on a live SSE stream (`FREECODE_HEADER_TIMEOUT_MS` / `FREECODE_SSE_STALL_TIMEOUT_MS`, `0` disables). Do NOT move this back above `normalizeAiSdkStream` — it drops `tool-input-delta`, so a large tool call looks like a dead stream. `rollout/trace.ts` (pure fold → spans; `in_flight` vs `hung` past `HANG_THRESHOLD_MS`) + `trace-render.ts` + `rollout/otlp.ts` (OTLP/HTTP JSON, no SDK dep, exported from the log not the hot path). CLI: `freecode trace [id] [--follow|--slow|--list|--json|--otlp]` — **operator reference is `TRACE.md` at the repo root** (flag interactions, how to read the verdict line). **A refused tool call is `function.denied` → `Trace.deniedSpans`, never `toolSpans`** (spec §5.1): `loop.ts` returns before `recordFunctionCall`, so before this event a mode-blocked call left no trace at all and a model looping against a mode it cannot satisfy folded to "did nothing". All four deny sites go through one `denyToolCall()` exit. `toolSpans` means **tools that ran** and its seven consumers depend on that, so denials stay out of it — which is also why an eval's `forbidTools` cannot see a refusal, and must be paired with an `expectTool` or it asserts nothing. **Cost**: `providers/pricing.ts` (USD/Mtok keyed `provider/model`, `~/.freecode/pricing.json` overrides, `PRICES_AS_OF` vintage) — an unknown model prices as `undefined`, never 0 or a near-miss, and a cache read is a **discount off the inclusive `inputTokens`, not an addend**. OTLP root span is `invoke_agent` with `gen_ai.conversation.id` on every span; `attrs()` rounds numerics to ints except the explicit `FRACTIONAL` set (cost, score) — adding a rate outside that set silently reports 0.5 as 1. |
+| **Observability** | Spec `2026-08-10-agent-observability.md`. Model-call events (`model.request`/`first_token`/`response`/`error`) recorded by `rollout/recorder.ts` around `callProviderOnce`; `model.request` is written *before* the call so an unterminated request is itself the evidence of a hang. **Timeouts live at the fetch layer** (`providers/fetch-timeout.ts`, wired as the `fetch` option on every provider): 300s for response headers, 180s of silence on a live SSE stream (`FREECODE_HEADER_TIMEOUT_MS` / `FREECODE_SSE_STALL_TIMEOUT_MS`, `0` disables). Do NOT move this back above `normalizeAiSdkStream` — it drops `tool-input-delta`, so a large tool call looks like a dead stream. `rollout/trace.ts` (pure fold → spans; `in_flight` vs `hung` past `HANG_THRESHOLD_MS`) + `trace-render.ts` + `rollout/otlp.ts` (OTLP/HTTP JSON, no SDK dep, exported from the log not the hot path). CLI: `freecode trace [id] [--follow|--slow|--list|--json|--otlp]` — **operator reference is `TRACE.md` at the repo root** (flag interactions, how to read the verdict line). **A refused tool call is `function.denied` → `Trace.deniedSpans`, never `toolSpans`** (spec §5.1): `loop.ts` returns before `recordFunctionCall`, so before this event a mode-blocked call left no trace at all and a model looping against a mode it cannot satisfy folded to "did nothing". All four deny sites go through one `denyToolCall()` exit. `toolSpans` means **tools that ran** and its seven consumers depend on that, so denials stay out of it — which is also why an eval's `forbidTools` cannot see a refusal, and must be paired with an `expectTool` or it asserts nothing. **Cost**: `providers/pricing.ts` (USD/Mtok keyed `provider/model`, `~/.freecode/pricing.json` overrides, `PRICES_AS_OF` vintage) — an unknown model prices as `undefined`, never 0 or a near-miss, and a cache read is a **discount off the inclusive `inputTokens`, not an addend**. OTLP root span is `invoke_agent` with `gen_ai.conversation.id` on every span; `attrs()` rounds numerics to ints except the explicit `FRACTIONAL` set (cost, score) — adding a rate outside that set silently reports 0.5 as 1. **Prompt-cache invariant (RC8, fixed 2026-09-06)**: mutable per-turn prompt state — memory recalls, todo block, drained `<system-reminder>`s — rides `ExecuteOptions.ephemeralTail`, a final user message appended AFTER `applyMessageCaching` places its anchors; it must never go in the system param (system precedes every message, so one changed byte re-sends the whole conversation) and must never carry a breakpoint. Only the compaction summary may be a mutable system block, because compaction documents its own invalidation. The D2 miss detector (`providers/cache-miss.ts`) holds an undocumented miss one sample and acquits it if the next read recovers to the pre-miss boundary — implicit caches (MiniMax) blip without a rewrite. `FREECODE_EPHEMERAL_TAIL=0` reverts placement, for `eval ab` measurement only. See `docs/caching-architecture.md` §1.1 + cache-observability spec §D2.1. |
 | **Thread Store** | `store/thread-store.ts`, `store/sqlite-store.ts`, `store/json-store.ts`, `store/remote.ts` |
 | **Sessions** | `session/manager.ts`, `session/store.ts`, `session/prompt.ts`, `session/end-session.ts` |
 | **Compaction** | `compaction/service.ts`, `compaction/selector.ts`, `compaction/summarizer.ts`, `compaction/tokens.ts` |

TODO.md

@@ -975,3 +975,31 @@ page's Known gaps.
 - [ ] **`anthropic` is the only provider with an OAuth mode.** `freecode auth
  login` rejects any other provider by name. Fine today — no other catalogue
  entry has a subscription surface freecode can reach.
+
+## Findings (ephemeral-tail cache fix — 2026-09-06)
+
+RC8 in the token-efficiency spec: memory/todo/reminder session system blocks
+rewrote the cached prefix every inner-loop turn; moved to
+`ExecuteOptions.ephemeralTail` (final user message, appended after the cache
+anchors). Detector gained a one-sample deferral for provider blips
+(cache-observability spec §D2.1). What remains open:
+
+- [ ] **A full provider-side eviction still alarms as a rewrite.** D2.1 acquits
+ a miss whose next read recovers to the pre-miss boundary; a miss where the
+ read never recovers (upstream evicted everything) is indistinguishable
+ from a real rewrite by usage numbers alone and produces the same warning.
+ `FREECODE_DEBUG_CACHE=1` segment hashes are the manual tiebreak.
+- [ ] **The UserPromptSubmit hook no longer sees memory/todo/reminder text.**
+ The hook rewrites the joined *system* prompt, and those blocks are message
+ content now. No known hook depended on them; if one surfaces, the hook
+ contract needs a decision (expose the tail read-only, or accept the loss).
+- [ ] **`FREECODE_EPHEMERAL_TAIL=0` should eventually be deleted.** It existed
+ so `eval ab` could price the two placements; the ledger entry
+ (`2026-09-05-redirect-1`) is decided "kept", so the old placement is now
+ dead code behind an env flag. Delete the flag, its `VARIABLE_ENV_KEYS`
+ row, and the `!tailEnabled` branches in `loop.ts` together.
+- [ ] **Watch: do tail-placed todo nudges lengthen tedious runs?** In the A/B,
+ `count-something-tedious` ran 22 candidate turns vs 11 baseline (one
+ spiral-by-design case, 3 trials — could be variance). If long-run turn
+ counts creep after this change, the nudge's salience as the final user
+ message is the first suspect.

apps/core/src/agent/loop.ts

@@ -1133,6 +1133,7 @@ export class AgentLoop {
  provider: string,
  model: string | undefined,
  context: { tree: string; gitHead: string; clock: string },
+ ephemeralTail: string,
  ): Promise<Awaited<ReturnType<typeof this.sendToProvider>>> {
  if (!isContextOverflowError(error)) throw error;
 
@@ -1168,6 +1169,7 @@ export class AgentLoop {
  provider,
  model,
  context,
+ ephemeralTail,
  );
  }
 
@@ -1518,10 +1520,24 @@ export class AgentLoop {
  // gate) into this turn's prompt. Transient — never persisted to history.
  const reminderText = this.pendingReminders.join("\n\n");
  this.pendingReminders = [];
- // Session-only system blocks: todo state and transient reminders. They
- // change, but they sit at the tail of the system array and the message
- // anchors that actually drive cache reads are downstream — so even a
- // full rewrite here does not touch the cached static prefix.
+ // Session-only system block: the compaction summary alone. It changes
+ // only when compaction runs, and compaction already documents its
+ // invalidation — so the system param stays byte-stable between
+ // compactions.
+ //
+ // Memory / todos / reminders used to sit here too, which was the D2
+ // "unexpected_creation" bug: system precedes every message, so any
+ // change to these between inner-loop requests re-sent the ENTIRE
+ // conversation at full price (reads collapsed to the static prefix).
+ // They now ride as `ephemeralTail` below — appended as a final user
+ // message AFTER the cache anchors (generic-provider), where a change
+ // costs only its own tokens. Same architecture as Claude Code's
+ // <system-reminder> injection.
+ // Measurement escape hatch (same pattern as FREECODE_DISABLE_REDIRECT):
+ // `FREECODE_EPHEMERAL_TAIL=0` reverts to the pre-fix system-block
+ // placement so `eval ab` can price the two side by side. Re-read every
+ // turn — the ab runner flips it per side after boot.
+ const tailEnabled = process.env.FREECODE_EPHEMERAL_TAIL !== "0";
  const sessionBlocks = [
  ...(compactionSummary
  ? [
@@ -1531,16 +1547,28 @@ export class AgentLoop {
  },
  ]
  : []),
- ...(memoryBlock ? [{ text: memoryBlock, cache: false }] : []),
- ...(todoBlock ? [{ text: todoBlock, cache: false }] : []),
- ...(reminderText ? [{ text: reminderText, cache: false }] : []),
+ ...(!tailEnabled && memoryBlock
+ ? [{ text: memoryBlock, cache: false }]
+ : []),
+ ...(!tailEnabled && todoBlock
+ ? [{ text: todoBlock, cache: false }]
+ : []),
+ ...(!tailEnabled && reminderText
+ ? [{ text: reminderText, cache: false }]
+ : []),
  ];
+ const ephemeralTail = tailEnabled
+ ? [memoryBlock, todoBlock, reminderText]
+ .filter((s) => s && s.length > 0)
+ .join("\n\n")
+ : "";
  const blocks = [...systemBlocks, ...sessionBlocks];
 
  // UserPromptSubmit Hook — can modify the joined system before send.
- // Must not collapse static + session into one cache:true blob (that
- // puts todos/memory/reminders under the breakpoint). See
- // apply-system-hook.ts.
+ // Must not collapse static + session into one cache:true blob. The
+ // ephemeral tail (todos/memory/reminders) is deliberately NOT part of
+ // what the hook sees: it is per-request message content now, not system
+ // prompt. See apply-system-hook.ts.
  const joinedSystem = blocks.map((b) => b.text).join("\n\n");
  const hookResult = await this.hooks.runUserPromptSubmit(joinedSystem, {
  sessionId: this.state.sessionId,
@@ -1575,6 +1603,7 @@ export class AgentLoop {
  provider,
  model,
  context,
+ ephemeralTail,
  );
  this.overflowCompactions = 0;
  } catch (error) {
@@ -1584,6 +1613,7 @@ export class AgentLoop {
  provider,
  model,
  context,
+ ephemeralTail,
  );
  }
 
@@ -1835,6 +1865,9 @@ export class AgentLoop {
  gitHead: string;
  clock: string;
  },
+ // Mutable per-turn state (memory/todos/reminders), appended after the
+ // cache anchors — see ExecuteOptions.ephemeralTail.
+ ephemeralTail = "",
  ): Promise<{
  content: string;
  thinking?: string;
@@ -1857,6 +1890,7 @@ export class AgentLoop {
  system,
  p === provider ? model : undefined,
  context,
+ ephemeralTail,
  ),
  { sessionId: this.state.sessionId, signal: this.abort.signal },
  );
@@ -1870,6 +1904,8 @@ export class AgentLoop {
  model: string | undefined,
  // Required so the dynamic user-message prepend has the file tree + clock.
  context: { tree: string; gitHead: string; clock: string },
+ // See ExecuteOptions.ephemeralTail — appended past the cache anchors.
+ ephemeralTail = "",
  ): Promise<{
  content: string;
  thinking?: string;
@@ -1958,7 +1994,8 @@ export class AgentLoop {
  model: resolvedModel,
  messageCount: prunedMessages.length,
  toolCount: tools.length,
- promptChars: estimatePromptChars(prunedMessages, system),
+ promptChars: estimatePromptChars(prunedMessages, system) +
+ ephemeralTail.length,
  streamed: Boolean(aiProvider.stream),
  });
 
@@ -2009,6 +2046,7 @@ export class AgentLoop {
  effort: this.state.effort,
  abortSignal: this.abort.signal,
  sessionId: this.state.sessionId,
+ ephemeralTail: ephemeralTail || undefined,
  })) {
  if (ttft_ms === undefined) {
  ttft_ms = Date.now() - startedAt;
@@ -2120,6 +2158,7 @@ export class AgentLoop {
  effort: this.state.effort,
  abortSignal: this.abort.signal,
  sessionId: this.state.sessionId,
+ ephemeralTail: ephemeralTail || undefined,
  });
 
  this.emitCacheWarm(result.usage);

apps/core/src/agent/max-iterations.test.ts

@@ -41,13 +41,16 @@ test("a run that never stops calling tools gets a graceful wrap-up, not a bare c
  const provider = "maxiter-fake";
  const FINAL_TEXT = "Finished the parser; wiring is still outstanding.";
 
- const calls: Array<{ system: SystemBlock[] }> = [];
+ const calls: Array<{ system: SystemBlock[]; ephemeralTail?: string }> = [];
  registerProvider(provider as ProviderId, {
  info: info(provider),
  create: (): AIProvider => ({
  info: info(provider),
- execute: async ({ system }): Promise<ExecuteResult> => {
- calls.push({ system: Array.isArray(system) ? system : [] });
+ execute: async ({ system, ephemeralTail }): Promise<ExecuteResult> => {
+ calls.push({
+ system: Array.isArray(system) ? system : [],
+ ephemeralTail,
+ });
  // Always emits a (bogus) tool call, so the loop never stops itself —
  // the only thing that can end this run is the iteration cap.
  return {
@@ -86,12 +89,14 @@ test("a run that never stops calling tools gets a graceful wrap-up, not a bare c
  // The model's last real text survives, not a bare status string.
  assert.match(result.content ?? "", new RegExp(FINAL_TEXT.replace(/[.]/g, "\\.")));
  assert.match(result.content ?? "", /iteration safety limit/);
- // The final turn's prompt carried the wrap-up nudge.
+ // The final turn's prompt carried the wrap-up nudge — in the ephemeral
+ // tail, not the system param: reminders are per-request message content
+ // so their churn cannot invalidate the cached prefix.
+ assert.match(calls[2].ephemeralTail ?? "", /Do not call any more tools/);
  const lastSystem = calls[2].system.map((b) => b.text).join("\n");
- assert.match(lastSystem, /Do not call any more tools/);
+ assert.doesNotMatch(lastSystem, /Do not call any more tools/);
  // Earlier turns were not nudged yet.
- const firstSystem = calls[0].system.map((b) => b.text).join("\n");
- assert.doesNotMatch(firstSystem, /Do not call any more tools/);
+ assert.doesNotMatch(calls[0].ephemeralTail ?? "", /Do not call any more tools/);
  } finally {
  await runtime.dispose();
  }

apps/core/src/agent/redirect/loop-redirect.test.ts

@@ -44,7 +44,11 @@ const info = {
  supportsTools: true,
 };
 
-/** Every system prompt the loop sent, so the test can look for the advice. */
+/**
+ * Every prompt the loop sent (system + ephemeral tail, where redirect advice
+ * now rides as per-request message content), so the test can look for the
+ * advice.
+ */
 const systemsSeen: string[] = [];
 /** Every supervisor prompt, so the test can prove the evidence was passed. */
 const supervisorPrompts: string[] = [];
@@ -75,7 +79,9 @@ registerProvider("redirect-fake" as ProviderId, {
  stream: async function* (
  opts: ExecuteOptions,
  ): AsyncGenerator<ProviderChunk> {
- systemsSeen.push(systemText(opts.system));
+ systemsSeen.push(
+ [systemText(opts.system), opts.ephemeralTail ?? ""].join("\n"),
+ );
  yield {
  type: "tool_call",
  id: `call-${systemsSeen.length}`,

apps/core/src/eval/ab.ts

@@ -37,6 +37,7 @@ export class AbError extends Error {}
  * FREECODE_DISABLE_MEMORY_CONSOLIDATION `shouldConsolidate`, every call
  * FREECODE_BASH_COMPRESS `maybeCompressOutput`, every tool call
  * FREECODE_READ_LINE_NUMBERS read's `execute`, every call
+ * FREECODE_EPHEMERAL_TAIL `executeTurn`, every iteration
  *
  * A startup-read var (a provider key, a config path, a fetch timeout baked into
  * the client at `createTimeoutFetch`) would be swapped into `process.env` and
@@ -52,6 +53,7 @@ export const VARIABLE_ENV_KEYS = [
  "FREECODE_DISABLE_MEMORY_CONSOLIDATION",
  "FREECODE_BASH_COMPRESS",
  "FREECODE_READ_LINE_NUMBERS",
+ "FREECODE_EPHEMERAL_TAIL",
 ] as const;
 
 /**