Chore/v1 p0 blockers - #28
Merged
Merged
Conversation
The README pitched FreeCode as a tool that "drives AI coding assistants via browser automation" with a two-phase "ask which files, then send them" flow. That is the legacy Playwright path CLAUDE.md marks as not wired into the primary path. The product is a ~198-provider API agent loop. Rewritten against source: the real installer, all 16 tools, the command list, the four-frontend architecture, and the generic provider driver. The browser path stays as a blockquote noting it exists and is not the default. Three claims were corrected while writing: `freecode session` is list + delete only, `freecode memory` is graph stats/rebuild/UI, and `freecode update` is injected by the TUI (apps/tui/src/entry.ts:194), not registered in create-cli.ts. Carries the subscription-auth warning callout too — it sits inside the rewritten body, and the docs page it links to lands later in this branch. beforeStable.md P0 #1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uX
Three ways a headless run diverged from an interactive one: - **No hooks.** HookSettingsManager and registerRtkHook were constructed only in startServer(), so `freecode run` loaded no settings.json hooks — the formatter that fires after every edit interactively silently did not fire in CI, same repo, two behaviours. Both now live in hooks/bootstrap.ts (initHooks), called by startServer() and the run handler. Only `serve` passes watch: true; a one-shot run exits before a settings edit could apply and would otherwise hold an fs watcher open past its last turn. - **`build` mode denied every write.** askPermission rejects immediately with no frontend listening and promptForPermission maps that to deny, while build's default for mutating tools is ask. So `freecode run "fix the test"` read fine and was denied every edit. `--yes` (-y) answers the ask tier with allow; repeatable `--allow <rule>` adds in-memory session grants. Both ride in on new AgentLoopConfig.autoApproveAsks / .sessionGrants. Scoped to the ask tier deliberately: a deny rule and a read-only mode still refuse, because those are decisions someone already made, not questions waiting for an answer. - **`--agent` was an unchecked cast**, so `--agent buld` fell through modeDefault's default branch and ran with build semantics. Added yargs .choices(), as `mcp add`'s type already has. Tests: agent/headless-permission.test.ts (5 — the first pins that the unattended default is still deny, two land a real write on disk via --yes and --allow, two assert --yes beats neither a deny rule nor plan mode) and hooks/bootstrap.test.ts (2). beforeStable.md P0 #2. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uX
`config.get` returned readConfig() verbatim, API keys and web-session cookies
included. The full JSON-RPC surface is reachable over web-server.ts's POST
/api — token-gated and loopback by default, but `host` is a parameter, so one
`--host 0.0.0.0` turned a debug convenience into key exfiltration.
redactConfig() builds the safe view field by field: { hasApiKey, model?,
authMode? } per provider entry, { hasCredential } per web entry, with current,
lastAgentMode and recovery passed through. An allowlist rather than a blocklist
of known secret names — WebCredentials has grown a secret-bearing field twice,
and a blocklist is wrong the day it grows a third.
No consumer lost anything. The only reader of the raw shape was the TUI's
getConfig() wrapper, which has no call sites; its ConfigInfo type now matches.
Every "is this provider set up" question already went through providers.list's
hasApiKey, which never carried the key.
Tests: providers/config-redaction.test.ts (4 — no secret from any block
survives serialization, the exact kept shape, an anonymous web session reads as
hasCredential: false rather than a missing block, an empty config stays empty).
beforeStable.md P0 #3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uXIt removed all of ~/.freecode on one `y` — sessions, rollout logs, memory, prompt history, usage — and the prompt did not say so. scripts/uninstall.sh already drew the right line (--purge for a full wipe, data kept otherwise); the CLI command was the copy that got it wrong, so the fix is to make them agree rather than invent a third behaviour. - Default takes the launcher and ~/.freecode/builds — the program. Everything else stays. - `--purge` takes the directory, and the line it prints names what is inside it rather than the path alone. - `--dry-run` added; `--force` gained -y/--yes so both entry points accept the same flags. No `--keep-data`: keeping data is now the default, and a flag for the default reads like the other behaviour is still lurking. Two things fell out. A non-TTY stdin resolved readline.question at once with an empty answer, which the old code read as "no" — correct by accident, and silent; it is now an error naming --force. And the isDirectory() branch before rmSync/unlinkSync is gone, since rmSync with recursive covers both. Tests: cli/commands/uninstall.test.ts (5, against a pure planUninstall() so the safety rule is asserted without deleting anything). The separate gap that uninstall ignores FREECODE_HOME / FREECODE_INSTALL_DIR and misses the Windows launcher path is untouched and still tracked; it takes the fixed item's place in the CLI page's known gaps. beforeStable.md P0 #4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uX
§0.1 of the OAuth spec was the risk section and had no resolution. A v1 that ships subscription auth as a headline feature needs an explicit, user-visible stance — not a spec paragraph. Resolved as: ship it, state the risk plainly, do not euphemize. Recorded in the spec as §0.2 and, more importantly, published: - /getting-started/anthropic-subscription — a new page whose second section is the risk, before any instructions. FreeCode presents itself to Anthropic as Claude Code; Anthropic reserves subscription inference for its own surfaces and has acted against tools doing this; the account at risk is the user's. Then the three opt-ins and the narrow fourth (no key configured plus a freecode login — an imported Claude Code login deliberately does not count), the login flow, status/logout, the org-forbidden 403 latch and API-key fallback, and why cost is stamped on the call. - /reference/cli gained a `freecode auth` section; the command was undocumented entirely. - /reference/env gained FREECODE_ANTHROPIC_AUTH, flagged as an opt-in rather than a neutral switch. The code needed nothing: auth.ts already printed the §0.1 disclosure on login and the identity block was already quarantined to the OAuth path with a test. The gap was that a user could only read any of it after deciding to run the command. Not adopted: a y/N confirmation on login. §0.1 chose "print once, no repeated nagging" deliberately, and a prompt in front of a command the user just typed by name buys nothing the paragraph above it does not already say. Three known gaps found while writing (unmapped tool names, single account, anthropic-only OAuth) are recorded in TODO.md. beforeStable.md P0 #5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uX
beforeStable.md is the 2026-09-05 audit against main at 360d1e5 (v0.29.0): health checks, the five P0 blockers, six P1s, what is explicitly not a blocker, and the path to 1.0. Each P0 now carries what was actually done and why, so the reasoning survives the branch. All five P0 items are closed. What remains before the tag is `pnpm eval:gate` with FREECODE_JUDGE_PROVIDER set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uX
`tearDownIfEmpty` disposed a session's ring buffer the moment its last
subscriber left — which, for a single browser, is every disconnect. A
reconnect carrying Last-Event-ID then found no record and was told
`{ gap: false, events: [] }`: nothing was missed. Everything produced
while away was lost, without even a `stream_gap` marker. Strictly worse
than eviction, which at least admits the loss.
Record lifetime is now decoupled from subscriber count. The last leave
starts a 5-minute TTL (`emptySince`) that the existing reaper enforces;
a reconnect inside the window clears it and replays for real. Events
emitted while nobody is attached keep being buffered, as they already
were.
`replayForSubscriber` also stops lying when the record is genuinely
gone: a client that claims to have seen events but finds no buffer now
gets a gap, since we cannot know what it missed.
Tests: 4 in `web/stream-subscribers.test.ts` — replay across a full
disconnect, a gap once reaped, survival of a partial disconnect, and a
reconnect clearing the TTL. `runReaperForTests(now?)` is the seam; a
five-minute TTL is not assertable against a 15s unref'd interval.Two failures that compounded in the same place — the summary that replaces most of a long session. **The brief was compacted first.** `selectForCompaction` preserved only the tail and summarized `messages.slice(0, firstPreservedIndex)`, so the oldest content went first and the first user message went with it. On the next compaction the *summary of it* was summarized again: the founding instruction decayed faster than anything else in the window, which is the plausible mechanism behind long-session drift off the task. The first user message is now carved out and preserved verbatim, after tail-trimming so trimming can never evict it. A head over `maxPreserveHeadTokens` (2k) is a pasted document rather than an instruction and is summarized as before. **The summary never saw the work.** A tool-calling turn was recorded as the stub `[Executed N tools]`, so the transcript handed to the summarizer contained none of the edits, commands or errors that were the actual work — and `summarizer.ts`'s `extractToolCalls`/`extractFiles` matched nothing, two paths that could not fire. `compaction/tool-transcript.ts` renders one bounded line per call (`Tool <name>: <args> -> <outcome>`, in the format the summarizer already greps for), and `addToolTurn` records it. The turn budget moved with it. `MemoryService.normalizeContent` is gone: it clipped the *tail* of a long message, i.e. the most recent tools. The transcript drops the oldest calls instead and says how many, which is the right end to lose. `maxToolOutputChars` is now that turn budget (4k). Tests: 6 in `tool-transcript.test.ts` (incl. one pinning the format against the summarizer's own regexes), 2 in `selector.test.ts` (the brief survives two compactions; an oversized head does not), and the two existing service/pruning tests rewritten to the new invariants. 1234 pass / 0 fail.
`CLAUDE.md` calls `METHODS` the source of truth for the JSON-RPC surface. It declared 25 of the 50 implemented handlers, so all of `memory.*`, `config.*`, `models.*` and eight session ops had zero compile-time checking in frontends — a wrong param shape was a runtime discovery. `METHODS["session.send"]` also declared `StreamResponse` as its result, which it has never returned: the handler resolves with the loop's result, and per-token output goes over the stream channel. Its params were missing `model`, `effort` and `agentMode` too. All 25 are now declared, `session.send` is corrected, and `ipc/methods-coverage.test.ts` asserts the two sets are equal in both directions — adding a handler without declaring it fails the suite, which is what makes the source-of-truth claim true rather than aspirational. Six result shapes needed wire types, since `packages/shared` cannot import from an app: `MemoryEntry`/`MemoryType`, `MemoryGraphStats`, `RedactedConfig`, `TurnResult`, `ExportedSession` and `ModelInfo`. They follow the existing `SessionMeta`/`SerializedMessage` mirror pattern, and `ipc/wire-shapes.test.ts` pins each one with a type-level assignability assertion so drift is a `check-types` failure rather than a field that silently stops reaching frontends. Two deliberate narrowings: `TurnResult` omits `LoopResult.finalState` (the loop's internal state machine — no frontend reads it), and there is no wire type for the raw config, only the redacted one. 1237 pass / 0 fail; check-types clean across all workspaces.
Every handler reads its params through `params as { … }`, a cast that
checks nothing. A missing or misspelled field arrived as `undefined`,
failed somewhere inside the handler, and came back as -32603 — an
internal error, which tells the caller the server is broken when the
request was.
`REQUIRED_PARAMS` in packages/shared declares, per method, the params a
handler genuinely cannot proceed without and their JSON types.
`handleRequest` checks it before dispatch and answers -32602 naming the
field and both types, so the call can be fixed from the error alone.
The table is typed `Record<MethodName, …>`, so a new method without an
entry is a compile error — there is no way to add a method that quietly
skips validation.
Scope is deliberately narrow. It validates presence and type of REQUIRED
params only: it is not a schema validator. Optional params stay the
handler's business (they have defaults), `null` counts as missing because
that is how a JSON caller usually spells "nothing" and it fails the same
way, and unknown params are ignored so a newer frontend still talks to an
older core. The rule for what goes in the table is what the handler
actually needs, not what `METHODS` types — `commands.list` types
`projectPath` as required but falls back to cwd, so validation must not
reject a call the handler would have served.
Tests: 9 in `ipc/validate-params.test.ts`, including the point of the
exercise — `handleRequest` returning -32602 rather than -32603 — plus one
that an unknown method is still -32601. 1246 pass / 0 fail.`settings.json` is hand-edited, security-relevant, and had neither. Four modules read it independently — permissions, hooks, memory, redirect — and each ignores what it does not recognise. Right per reader; the sum was that `"permission"` for `"permissions"` disabled every rule and looked exactly like the permission system being broken. `settings/known-keys.ts` is the one place that knows the whole shape, so it can say "unknown key" and, for a near miss, which key was meant. `settings/validate.ts` runs it over both scopes once per process, from the shared hook bootstrap — the same bootstrap, because a second call site is how `serve` and `run` diverged last time. Deliberately a NAME check, not a schema validator. Values stay each reader's business: they already validate and default their own, and a second copy of those rules would diverge. Hook event names are left to `hooks/settings.ts`, which already reports an unknown one with the full valid list — one good message beats two in different words. And it warns rather than refuses: a file with a stray key is still a usable file. `schemas/settings.schema.json` is the editor half, wired up with `$schema` (the one key FreeCode ignores on purpose). Two descriptions of the same file drift the moment a key lands in only one, so a test asserts the schema's key set matches `KNOWN_SETTINGS` section by section. Tests: 9 in `settings/known-keys.test.ts`, including the motivating typo, a section-level typo, the two deliberate non-warnings, and the schema/runtime drift guard. 1255 pass / 0 fail. Docs: `/reference/settings` gained the `$schema` section, `redirect` in the top-level table (it was reachable but undocumented there), and the four memory keys the page never listed —`retrievalJudge`, `autoConsolidate`, `consolidateMinHours`, `consolidateMinSessions`. Two Known-gaps bullets and their `TODO.md` entries are now closed.
All six P1 items are closed. Each entry says what was wrong, what was built, and — where it matters more — what was deliberately not built: no shared settings loader (the three-merge-rules gap stays open and is not what made a typo invisible), no wire type for the raw config, no validation stricter than the handler it guards. Step 5 of the path to 1.0 is now the eval gate. P1 #2 and #3 both change what the model sees after a compaction, so that run is the measurement, not a formality.
I wrote that a moved judged suite would point at the compaction changes. That is wrong: compaction fires at ~107k input tokens, the longest eval prompt is ~65 tokens, and every case runs a single turn — so no case has ever compacted or can. The transcript from P1 #3 reaches the model only through the summary (loop.ts:1446 reads .summary, not recentMessages), so with no compaction the prompt is byte-identical before and after. The gate is a pre-tag regression check here, not a measurement. Recorded the harness gap in TODO.md, with the cheap fix (a FREECODE_COMPACT_TARGET_TOKENS override in the runner) noted over the expensive one (multi-turn replay cases).
My TODO entry made this look like a new discovery. It is not: the harness already tracks it in two places — CATEGORIES_WITHOUT_CASES in dataset.test.ts and §9.1 of the case-registry spec, which states the objection precisely (compacting is an accident today, and an accident is not a p >= 0.99 case). A third copy in TODO.md would drift from both, so that entry is now a pointer. The idea itself belongs in §9.1 and is recorded there: the compaction threshold is already a knob (FREECODE_COMPACT_TARGET_TOKENS via getCompactTarget), so lowering it for one case turns compacting from an accident into the point of the case. It needs a per-case env key on EvalCase, which is harness work the spec says should be specified before it is built — including whether arbitrary env is allowed (it should not be) and how the gate compares a case that runs in a different environment. Not building it. Recording it where the decision will be made.
All eleven items (five P0, six P1) are closed and their outcomes live in the commits that made them, so the audit file was a duplicate record with a shelf life. `pnpm eval:gate` opened all three suites against anthropic/claude-opus-5 — the last step it was tracking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uX
… temperament
The case exists to assert that the model reads before opining — its
`whyModelBacked` says so. What it actually blocked on was `expectMaxTurns: 6`,
a budget calibrated against minimax/MiniMax-M3, the only model it had ever run
on (18/18 passes). Its first run on claude-opus-5 came in at 9, 5 and 10 turns
and failed the majority.
The trace says that is thoroughness, not thrashing: 17 tool calls over 9 turns,
`repeatedCalls: 0`, no oscillation, monotonically widening from the named file
to its consumers to the tests and spec, ending in a real answer. The prompt
("what else would that affect") has no natural stopping point, so turn count
here measures how much a model explores, which differs per model — while the
gate's own regression rule is already per-model.
So: assert the sequencing directly with `expectFirstToolIn: ["read"]`, which is
model-independent, and raise the budget to 12 so it still catches a loop.
Note `expectMaxTurns` is checked before the tool assertions
(`scorers/trajectory.ts:29`), so the old failure never evaluated whether `read`
fired first — the trace confirms it did.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cy5ynPzdM4gJ8o1qKpp5uXThe latest updates on your projects. Learn more about Vercel for GitHub.
|
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.