diff --git a/.gitignore b/.gitignore index a62f7b72..f3fd3129 100644 --- a/.gitignore +++ b/.gitignore @@ -10,3 +10,27 @@ __pycache__/ # Subagent-driven-development scratch workspace (ledger, briefs, review packages) .superpowers/ + +# S6, the ACS envelope log (N26). Local demo artifact; carries raw tool +# arguments unmodified, so it is never committed. +.acs/ + +# Scratch tree for packages/guardian/test/server.test.ts's outer-net test: an +# identical copy of packages/guardian/src, placed one directory deeper so its +# schema path resolves to nothing. The test removes it in a `finally`; this +# line only covers a run killed mid-test, so a stray copy of the Guardian's +# source can never be committed as if it were source. Also excluded from +# tsconfig.json, for the same reason: a killed run's leftover copy must not +# make `bun run typecheck` see a stray duplicate of the Guardian's own +# source. Fixed name (not mkdtemp's random one), so this one line covers it. +packages/guardian/tmp-schemaless-scratch/ + +# Same reasoning, two more scratch trees: server.test.ts's fake +# validate-envelope.ts test doubles (a non-string .message, and a throwing +# .message accessor), each alongside a real, unmodified copy of every other +# file in packages/guardian/src. Two separate directories, not one, because +# Bun's module cache keys by resolved path and these two fake sources differ +# from each other -- sharing one directory would silently hand the second +# test the first's already-loaded module. +packages/guardian/tmp-undefined-message-scratch/ +packages/guardian/tmp-throwing-message-accessor-scratch/ diff --git a/README.md b/README.md index eb344bd8..3793e486 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ One wire contract between agent hosts and policy runtimes, so governance integra Today every policy vendor writes a module per agent, and every agent waits for a module per vendor. Microsoft's [Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) ships four host packages with four different architectures — a Copilot CLI extension, subprocess hooks for Claude Code and Antigravity, an in-process plugin for OpenCode — and documents the capability divergence between them in its own READMEs. -This repository shows the other shape. A host implements [ACS](https://github.com/Agent-Control-Standard/ACS) once and is governable by any conformant runtime. A runtime implements ACS once and governs any conformant host. This slice (V1) wires one host — Claude Code — to AGT's policy engine running unforked, its stock Rego bundle deciding, entirely over the ACS wire. +This repository shows the other shape. A host implements [ACS](https://github.com/Agent-Control-Standard/ACS) once and is governable by any conformant runtime. A runtime implements ACS once and governs any conformant host. V1 wires one host — Claude Code — to AGT's policy engine running unforked, its stock Rego bundle deciding, entirely over the ACS wire. V2 makes that wire visible: every envelope crossing it is recorded to a log and rendered live by `bun run inspector`. ## What this proves @@ -15,11 +15,19 @@ This repository shows the other shape. A host implements [ACS](https://github.co | AGT's policy engine runs unforked, over the ACS wire | AGT's published policy library decides, used as shipped, at a pinned upstream commit, with no source changes (`agt.lock`, [`test/pin.test.ts`](test/pin.test.ts)) | | The collapse is structural, not incidental | The host adapter contains no AGT-specific code and the AGT bridge contains no host-specific code — verifiable by reading the file list, and enforced by [`test/invariants.test.ts`](test/invariants.test.ts) | +**Delivered in V2** — the Envelope Inspector. + +| Claim | How it is demonstrated | +|---|---| +| R5.1 — every hook firing is inspectable as an ACS envelope, in both directions, including envelopes that fail validation | The Guardian records every envelope crossing its wire to `.acs/envelopes.jsonl` before validation, and `bun run inspector` renders it live ([`test/envelope-log-sink-roundtrip.test.ts`](test/envelope-log-sink-roundtrip.test.ts), [`packages/guardian/test/envelope-log-sink-wiring.test.ts`](packages/guardian/test/envelope-log-sink-wiring.test.ts)) | +| R5.2 — an ACS-first reader can trace one action end to end without reading AGT source | The Inspector imports nothing from the Guardian or the AGT bridge and names neither AGT nor any host — enforced by two gates in [`test/invariants.test.ts`](test/invariants.test.ts) | + **Planned, not yet built** — the rest of the claim this project is working toward. None of the following exists yet, and there is no CI in this repository at all. | Claim | Slice | |---|---| -| AGT is completely expressible in ACS: a machine-checked mapping of all eight intervention points and five verdicts, with a round-trip conformance case per cell | V7 | +| A machine-checked mapping of all eight intervention points and five verdicts, with a round-trip conformance case per cell — every cell resolved, green where ACS v0.1.0 expresses AGT and red with a named reason where it does not. Four are already known red: the two model-call points have no v0.1.0 hook, and two attributes the Trace pillar marks required have no source on the wire | V7 | +| Which ACS profiles and pillars this implementation claims, and which it does not — the matrix is the declaration. Trace is a measured non-claim, not a silence | V7 | | The same policy governs two structurally different coding agents, with the second host costing zero added AGT code | V5 | | A scheduled harness run against AGT `main` catches upstream drift automatically | V8 | @@ -38,7 +46,7 @@ The shaping doc is authoritative for requirements, shapes, and the breadboard. T git clone --recurse-submodules https://github.com/afogel/ACS_reference_implementation ``` -## Quickstart (R7.1 — one command on a laptop) +## Quickstart (R7.1 — starts with one command on a laptop) Requires [`bun`](https://bun.sh) and the [Claude Code](https://docs.claude.com/en/docs/claude-code) CLI (`claude`) on your `PATH`. @@ -58,41 +66,67 @@ This constructs the AGT bridge once, against the pinned stock policy bundle (`po ``` Guardian listening at http://localhost:8787/acs +Envelope log (S6): .acs/envelopes.jsonl ``` Leave it running. `hosts/claude-code/acs-hook.ts` defaults to exactly this URL; override with `ACS_GUARDIAN_URL` if it's listening elsewhere. -**3. Wire the hook into Claude Code.** +**3. Start the Envelope Inspector.** In a second terminal, after the Guardian (which is what creates `.acs/`): + +```bash +bun run inspector +``` + +``` +Envelope Inspector — tailing .acs/envelopes.jsonl +Ctrl-C to stop. +``` + +Every ACS envelope crossing the Guardian's wire is printed here as it happens — request and response, with a decision badge on responses: + +``` +── #2 20:44:33.130 ← RESPONSE steps/toolCallRequest id=e491180d-60a8-4982-b693-e63771c00e2d +● DENY reason_codes=[destructive_shell_command_blocked] policy_references=[agt_stock#destructive_shell_command_blocked] +``` + +`bun run inspector -- --from-start` replays a session already recorded. **`.acs/envelopes.jsonl` records each envelope the Guardian parsed, unmodified: nothing stripped, nothing redacted. So it carries raw tool arguments** — it is gitignored for that reason and never committed. Full walkthrough, with the real captured output for a deny, an allow, and a schema-invalid envelope: [`docs/demos/v2-runbook.md`](docs/demos/v2-runbook.md). + +**4. Wire the hook into Claude Code.** ```bash mkdir -p .claude cp hosts/claude-code/settings.json .claude/settings.json ``` -This registers `hosts/claude-code/acs-hook.ts` as a `PreToolUse` hook for the `Bash` tool — the "one hook" of this slice's name. +This registers `hosts/claude-code/acs-hook.ts` as a `PreToolUse` hook for the `Bash` tool — the "one hook" of V1's name. -**4. Run Claude Code with the hook.** +**5. Run Claude Code with the hook.** ```bash claude ``` -Ask it to run a destructive shell command, e.g. *"Use the Bash tool to run exactly this command: `rm -rf /`"*. The tool call is blocked, with the real policy-engine reasoning surfaced in the transcript — not a canned string, the actual text AGT's stock policy engine produces when it evaluates the pattern it matched. That pattern list is this project's own configuration (`policy/lib/data.json`), not something AGT ships — the stock bundle carries no shell/command patterns of its own, only generic PII regexes; what's stock is the *deciding module* (`agt.patterns`) and the priority chain that consults it, per R2.1 (zero Rego authored). See the framing note in [`docs/demos/v1-runbook.md`](docs/demos/v1-runbook.md) before narrating this demo. Ask for something harmless (`ls -la`) in the same session and it runs normally. Full walkthrough and what to watch for: [`docs/demos/v1-runbook.md`](docs/demos/v1-runbook.md). +Ask it to run a destructive shell command, e.g. *"Use the Bash tool to run exactly this command: `rm -rf /`"*. The tool call is blocked, with the real policy-engine reasoning surfaced in the transcript — not a canned string, the actual text AGT's stock policy engine produces when it evaluates the pattern it matched. That pattern list is this project's own configuration (`policy/lib/data.json`), not something AGT ships — the stock bundle carries no shell/command patterns of its own, only generic PII regexes; what's stock is the *deciding module* (`agt.patterns`) and the priority chain that consults it, per R2.1 (zero Rego authored). See the framing note in [`docs/demos/v1-runbook.md`](docs/demos/v1-runbook.md) before narrating this demo. Ask for something harmless (`ls -la`) in the same session and it runs normally. Full walkthrough and what to watch for: [`docs/demos/v1-runbook.md`](docs/demos/v1-runbook.md); with the Inspector running you also see both envelopes as they cross the wire. -Steps 1–2 were run against this exact tree to write this README: `bun install` completes clean, and `bun run guardian` prints the line above. Steps 3–4 were verified the same way the project's own tests verify them — piping a Claude Code–shaped `PreToolUse` payload on stdin straight into the hook shim against a running Guardian: +Watch the Inspector, not just the transcript, if the deny does not appear: the model may decline to issue the tool call at all on its own judgment, in which case no hook fires and the envelope log stays empty. And if you are scripting this rather than watching it, use `echo rm -rf /` as the payload — it matches the same pattern at offset 5 and is inert if it ever did execute, whereas an unattended `rm -rf /` is only safe for as long as the hook works, which is the thing under test. + +**What was actually run against this tree to write this quickstart.** Steps 1–3 were run end to end: `bun install` completes clean, `bun run guardian` prints both lines above, and `bun run inspector` rendered every envelope quoted here — the badge line above is pasted from that run, not composed. Steps 4–5 were run twice, two different ways. + +First, by piping a Claude Code–shaped `PreToolUse` payload on stdin straight into the hook shim against a running Guardian — the same way the project's own tests verify it. The shim never executes the command; it only asks the Guardian for a decision: ```bash echo '{"session_id":"demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"rm -rf /"}}' \ | bun run hosts/claude-code/acs-hook.ts -# {"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"matched pattern ... at offset 0"}} +# {"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"matched pattern (?i)rm\\s+-[a-z]*r[a-z]*f[a-z]*\\s+/(?:\\s|$) at offset 0"}} ``` -which is exactly the JSON Claude Code's own hook protocol sends and expects back; running `claude` interactively for step 4 exercises the identical path through the real CLI. +Second, through the real `claude` CLI with `.claude/settings.json` installed — Claude Code spawning `acs-hook.ts` as an actual `PreToolUse` subprocess and honouring the decision. That run was **headless** (`claude -p '' --allowedTools Bash`), not an interactive TUI session, and the payload was `echo rm -rf /` rather than `rm -rf /`: the configured pattern matches the raw command string with no argv parse, so it is denied by the same rule at offset 5 while being inert if it ever did execute. An unattended `rm -rf /` is only safe for as long as the hook works, which is the thing under test. The interactive TUI session was not run, so nothing here describes how the TUI renders the block. See [`docs/demos/v2-runbook.md`](docs/demos/v2-runbook.md) for the full captured output of both. ### Verify ```bash -bun test # 97 tests across 12 files (96 pass, 1 skip), including the R3.2/R3.3 gates below +bun test # 169 tests across 17 files (168 pass, 1 skip), including the R3.2/R3.3 + # and R5.1/R5.2 gates below # the skip is the byte-identity check, which needs UPSTREAM_BUNDLE — see verify:pin bun run typecheck # whole-workspace strict TypeScript check, zero errors ``` @@ -101,7 +135,13 @@ bun run typecheck # whole-workspace strict TypeScript check, zero errors ## Status -V1 ("one host, one hook") is implemented: a Claude Code `PreToolUse` hook, a Guardian process serving ACS over HTTP, and AGT's unforked stock policy bundle deciding behind it — see the quickstart above and [`slices/v1/README.md`](slices/v1/README.md). R3.2 and R3.3 (no AGT vocabulary in the host adapter, no host *output* vocabulary in it either, and no host vocabulary in the AGT bridge) are enforced by [`test/invariants.test.ts`](test/invariants.test.ts), not left to inspection. Slices V2–V8 are shaped and sliced but not started; they are tracked as issues on the project board, each with a stacked pull request. +V1 ("one host, one hook") is implemented: a Claude Code `PreToolUse` hook, a Guardian process serving ACS over HTTP, and AGT's unforked stock policy bundle deciding behind it — see the quickstart above and [`slices/v1/README.md`](slices/v1/README.md). + +V2 ("Envelope Inspector") is implemented: the Guardian records every ACS envelope crossing its wire to `.acs/envelopes.jsonl`, and `bun run inspector` tails and renders it live — see [`slices/v2/README.md`](slices/v2/README.md) and [`docs/demos/v2-runbook.md`](docs/demos/v2-runbook.md). One boundary worth stating up front: a schema-invalid envelope surfaces as a JSON-RPC **error**, not a `deny` decision. `N27 denyOnInvalidEnvelope()`, which turns Guardian-side failures into honoured ACS decisions, is V3. + +Four of this project's architectural claims are enforced by [`test/invariants.test.ts`](test/invariants.test.ts) rather than left to inspection: R3.2 and R3.3 (no AGT vocabulary in the host adapter, no host *output* vocabulary in it either, and no host vocabulary in the AGT bridge), and R5.1 and R5.2 (the Inspector imports nothing from the Guardian or the AGT bridge, and names neither AGT nor any host). + +Slices V3–V8 are shaped and sliced but not started; they are tracked as issues on the project board, each with a stacked pull request. ## License diff --git a/bun.lock b/bun.lock index c6d7d495..925be594 100644 --- a/bun.lock +++ b/bun.lock @@ -45,6 +45,10 @@ "guardian": "workspace:*", }, }, + "packages/inspector": { + "name": "inspector", + "version": "0.1.0", + }, }, "packages": { "@types/node": ["@types/node@26.2.0", "", { "dependencies": { "undici-types": "~8.3.0" } }, "sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg=="], @@ -129,6 +133,8 @@ "host-adapter": ["host-adapter@workspace:packages/host-adapter"], + "inspector": ["inspector@workspace:packages/inspector"], + "json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="], "require-from-string": ["require-from-string@2.0.2", "", {}, "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw=="], diff --git a/docs/demos/v2-runbook.md b/docs/demos/v2-runbook.md new file mode 100644 index 00000000..8b87eded --- /dev/null +++ b/docs/demos/v2-runbook.md @@ -0,0 +1,494 @@ +# V2 demo runbook: the Envelope Inspector + +**The demo, in the slice's own words** (from `docs/shaping/acs-reference-impl-slices.md`): + +> Watch the ACS request and response JSON stream live while you work in Claude Code. + +This runbook is written from a real run against this tree. Every block below marked +"captured" is pasted from the actual terminal, not reconstructed. + +## What a viewer should watch for + +1. A third terminal, beside the Guardian and the agent host, prints every ACS envelope + as it crosses the wire — request and response, both directions, one entry each. +2. The rendered JSON is the envelope the Guardian parsed, printed unmodified and + re-indented. The sink strips no field, redacts nothing, and reorders nothing it + controls. It is not a byte-for-byte replay — see *S6 carries raw tool arguments* + below for what the JSON parse normalises before the sink ever sees it. +3. The decision badge makes the outcome legible without reading the JSON: `● DENY` or + `○ ALLOW`, with `reason_codes` and `policy_references` beside it. A response that + carried no decision renders as `✖ ERROR` on the same line — deliberately not a + decision badge, because in this slice a schema failure is an error and not a `deny` + (see *the honest boundary* below). +4. Nothing in the Inspector knows what produced the decision. It imports nothing from + the Guardian and names neither AGT nor any host — enforced by two gates in + [`test/invariants.test.ts`](../../test/invariants.test.ts), not left to inspection. + That is what makes R5.2 ("an ACS-first reader can trace one action end to end + without reading AGT source") a property of the code rather than a claim in prose. + +**Read the V1 runbook's two framing notes first.** [`docs/demos/v1-runbook.md`](v1-runbook.md) +records that the destructive-command pattern list is this project's own configuration +supplied as data, not something AGT ships, and that the handshake declares rather than +negotiates. Both still hold here; V2 changes neither. + +## Prerequisites + +- `bun` installed, `bun install` run once at the repo root. +- This repo cloned with its submodule. +- For the last section only: the [Claude Code](https://docs.claude.com/en/docs/claude-code) + CLI on your `PATH` (`claude`). + +## The three terminals, in order + +### Terminal 1 — the Guardian, first + +```bash +bun run guardian +``` + +Captured: + +``` +Guardian listening at http://localhost:8787/acs +Envelope log (S6): .acs/envelopes.jsonl +``` + +**Why this one starts first: it creates `.acs/`.** The sink calls +`mkdirSync(dirname(path), { recursive: true })` when the Guardian constructs it at boot +(`packages/guardian/src/envelope-log-sink.ts`), so the directory exists from the moment the +Guardian is up. The log file itself does not appear until the first envelope is written +— after the Guardian booted, `.acs/` existed and was empty. + +The Inspector tolerates a file that does not exist yet (`sizeOf` returns 0 for a missing +path), so starting them out of order does not break anything. Guardian-first is still +the order to teach, because it is the order in which the two artifacts come into +existence. + +The sink is opt-in at the library level and on by default in the CLI: `startGuardian` +records only when `envelopeLogPath` is passed, and `packages/guardian/src/main.ts` passes +it. Override the location with `ACS_ENVELOPE_LOG`. + +### Terminal 2 — the Inspector + +```bash +bun run inspector +``` + +Captured: + +``` +Envelope Inspector — tailing .acs/envelopes.jsonl +Ctrl-C to stop. +``` + +It follows the log the way `tail -f` does: it starts at the current end and prints what +arrives from now on. `ACS_ENVELOPE_LOG` and `--envelope-log ` both point it elsewhere; the +default matches the Guardian's default, so neither hardcodes the other's value. + +Output is coloured when stdout is a TTY and `NO_COLOR` is unset. Every capture in this +runbook was taken with stdout redirected to a file, which is why the pasted bytes carry +no ANSI escapes; on a TTY the badge lines carry colour (red, green, and yellow, per the +constants in `packages/inspector/src/render.ts` — read from the source, not captured +here). + +### Terminal 3 — drive a tool call + +The shim reads a Claude Code `PreToolUse` payload on stdin and writes the hook result on +stdout. It never executes the command; it only asks the Guardian for a decision, which +is why piping `rm -rf /` into it is safe. + +```bash +echo '{"session_id":"demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"rm -rf /"}}' \ + | bun run hosts/claude-code/acs-hook.ts +``` + +Captured, from terminal 3: + +``` +{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"matched pattern (?i)rm\\s+-[a-z]*r[a-z]*f[a-z]*\\s+/(?:\\s|$) at offset 0"}} +``` + +## The deny, as the Inspector rendered it + +Captured, from terminal 2, verbatim: + +``` +── #1 20:44:32.555 → REQUEST steps/toolCallRequest id=e491180d-60a8-4982-b693-e63771c00e2d +{ + "jsonrpc": "2.0", + "method": "steps/toolCallRequest", + "id": "e491180d-60a8-4982-b693-e63771c00e2d", + "params": { + "acs_version": "0.1.0", + "request_id": "e491180d-60a8-4982-b693-e63771c00e2d", + "timestamp": "2026-08-09T20:44:32.535Z", + "metadata": { + "agent_id": "claude-code", + "session_id": "6ccc72b8-a167-5573-b3ea-a310262ea93f" + }, + "payload": { + "tool": { + "name": "Bash" + }, + "arguments": { + "command": { + "value": "rm -rf /" + } + } + } + } +} + +── #2 20:44:33.130 ← RESPONSE steps/toolCallRequest id=e491180d-60a8-4982-b693-e63771c00e2d +● DENY reason_codes=[destructive_shell_command_blocked] policy_references=[agt_stock#destructive_shell_command_blocked] +{ + "jsonrpc": "2.0", + "id": "e491180d-60a8-4982-b693-e63771c00e2d", + "result": { + "type": "final", + "acs_version": "0.1.0", + "request_id": "e491180d-60a8-4982-b693-e63771c00e2d", + "decision": "deny", + "reasoning": "matched pattern (?i)rm\\s+-[a-z]*r[a-z]*f[a-z]*\\s+/(?:\\s|$) at offset 0", + "reason_codes": [ + "destructive_shell_command_blocked" + ], + "policy_references": [ + { + "policy_id": "agt_stock", + "rule_id": "destructive_shell_command_blocked" + } + ] + } +} +``` + +The header line is `── #seq time direction method id=`. `seq` is a counter +scoped to the Guardian process. The `id` is the JSON-RPC id, which is what pairs the two +entries — it is the only identifier present in both directions, so pairing survives two +hooks in flight at once. + +## The allow, for contrast + +```bash +echo '{"session_id":"demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"ls -la"}}' \ + | bun run hosts/claude-code/acs-hook.ts +``` + +Captured, from terminal 3: + +``` +{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"allow"}} +``` + +Captured, from terminal 2, verbatim: + +``` +── #3 20:44:41.150 → REQUEST steps/toolCallRequest id=0781250c-0f99-43ab-bc66-e79e5fead2d9 +{ + "jsonrpc": "2.0", + "method": "steps/toolCallRequest", + "id": "0781250c-0f99-43ab-bc66-e79e5fead2d9", + "params": { + "acs_version": "0.1.0", + "request_id": "0781250c-0f99-43ab-bc66-e79e5fead2d9", + "timestamp": "2026-08-09T20:44:41.139Z", + "metadata": { + "agent_id": "claude-code", + "session_id": "6ccc72b8-a167-5573-b3ea-a310262ea93f" + }, + "payload": { + "tool": { + "name": "Bash" + }, + "arguments": { + "command": { + "value": "ls -la" + } + } + } + } +} + +── #4 20:44:41.203 ← RESPONSE steps/toolCallRequest id=0781250c-0f99-43ab-bc66-e79e5fead2d9 +○ ALLOW +{ + "jsonrpc": "2.0", + "id": "0781250c-0f99-43ab-bc66-e79e5fead2d9", + "result": { + "type": "final", + "acs_version": "0.1.0", + "request_id": "0781250c-0f99-43ab-bc66-e79e5fead2d9", + "decision": "allow" + } +} +``` + +A clean allow carries no `reasoning`, no `reason_codes`, and no `policy_references`, so +the badge is bare. That absence is the signal: a policy that fired but let the action +proceed arrives as `allow` with a **non-empty** `policy_references`, and the badge +renders that case differently — `◐ ALLOW (policy fired)` — because rendering the two +identically is exactly what U21 exists to prevent. No policy-fired allow occurred in +this run; the rendering above is the deny and the clean allow, which are what the pinned +configuration actually produced. + +The label used to end `— ACS "warn"`, and it was retired by PR #11's review: ACS has no +`warn` disposition, so the badge was teaching one. The Inspector is an ACS-first reader +and R5.2 keeps it clear of policy-runtime vocabulary; a string on screen carries that +vocabulary further than an identifier does. The badge now says only what ACS says +happened — a policy fired, and the action was allowed. + +## The honest boundary: a schema-invalid envelope is an error, not a deny + +The request is recorded **before** validation, so an envelope that fails the schema is +visible rather than swallowed. What comes back, though, is a JSON-RPC **error** — not an +ACS `deny` decision. + +```bash +curl -s -X POST http://localhost:8787/acs \ + -H 'content-type: application/json' \ + -d '{"jsonrpc":"2.0","method":"steps/toolCallRequest","id":"1","params":{"acs_version":"0.1.0"}}' +``` + +Captured, from terminal 2, verbatim: + +``` +── #5 20:44:57.829 → REQUEST steps/toolCallRequest id=1 +{ + "jsonrpc": "2.0", + "method": "steps/toolCallRequest", + "id": "1", + "params": { + "acs_version": "0.1.0" + } +} + +── #6 20:44:57.829 ← RESPONSE steps/toolCallRequest id=1 +✖ ERROR -32010 ACS envelope failed schema validation at /params/request_id: must have required property 'request_id' +{ + "jsonrpc": "2.0", + "id": "1", + "error": { + "code": -32010, + "message": "ACS envelope failed schema validation at /params/request_id: must have required property 'request_id'", + "data": { + "pointer": "/params/request_id" + } + } +} +``` + +**This is a real limitation of V2, stated plainly.** `✖ ERROR -32010` is what a +Guardian-side failure looks like today. It is *not* a decision, so it is not something +the host's decision path honours the way it honours a `deny`. +`N27 denyOnInvalidEnvelope()` — the affordance that turns schema and bridge failures +into explicit ACS `deny` **decisions** — is **slice V3**, defined in +[`docs/shaping/acs-reference-impl-slices.md`](../shaping/acs-reference-impl-slices.md) +§V3, alongside `N6 applyFailurePosture()` and `N7 validateDecision()`. The Inspector is +where that change will become visible: the same request will come back with a badge +instead of an error line. + +A body that will not parse as JSON at all goes further: there is no request entry to +pair with, because there was never a parseable request. The Inspector renders the lone +response rather than hiding it. Captured from `curl -d 'this is not json'`, verbatim — +this one was taken against a freshly restarted Guardian, which is why its `seq` is `#1` +rather than continuing the run above: + +``` +── #1 20:51:47.882 ← RESPONSE (no method) (unpaired) +✖ ERROR -32700 Parse error +{ + "jsonrpc": "2.0", + "id": null, + "error": { + "code": -32700, + "message": "Parse error" + } +} +``` + +`(no method)` and `(unpaired)` are what the renderer prints for `method: null` and +`rpc_id: null`. Both are real states on this wire, so both are shown. + +## S6 carries raw tool arguments + +`.acs/envelopes.jsonl` records the JSON value the Guardian parsed, unmodified: no field +stripping, no redaction, no reordering of anything we control. Pretty-printing happens at +render time only. An inspector that showed something other than what was sent would be +worse than no inspector at all. + +**One honest qualifier, added by the whole-branch review.** "Unmodified" is a claim about +what *we* do, not a claim of byte identity. The sink is handed `await req.json()`, so a +JSON parse has already happened: duplicate keys are collapsed to the last one, number +literals are canonicalised (`1.0` renders as `1`, `1e2` as `100`), and integer-like +object keys are hoisted ahead of the rest and sorted. Tool argument names come from the +host, so an `arguments` object containing both `"0"` and `"a"` is a real possibility +rather than a contrived one — and it would render with `"0"` first regardless of send +order. If you need the literal bytes a host sent, capture them at the transport; S6 is +the parsed envelope. What S6 does guarantee is that nothing between the parse and the +file removed, rewrote, or hid any part of it — with two named exceptions, both on the +*stringify* side rather than the parse side, since a stringify happens every time S6's +line is written and again every time the Inspector pretty-prints it. A number literal +too large for a JS `double` (`1e400`) parses fine, as `Infinity`, but `JSON.stringify` +writes `Infinity` as `null` — the field survives; its value does not. `-0` survives the +parse with its sign intact and loses it the same way: `JSON.stringify(-0)` is `"0"`. +Neither is something this project decided; both are `JSON.stringify`'s own behaviour, +wherever it runs. + +The consequence is direct: **the log contains whatever your tool calls contained** — +file paths, command lines, and anything else that rode along in `arguments`. That is why +`.acs/` is in [`.gitignore`](../../.gitignore) and why it is never committed. It is a +local demo artifact. Treat it the way you would treat a shell history file. + +It also grows without bound. V2 builds no rotation; that is recorded as an accepted +limitation in the slices doc. + +## Clearing the log mid-demo + +```bash +: > .acs/envelopes.jsonl +``` + +Truncate in place. The Inspector picks up cleanly afterwards without a restart, because +`tailEnvelopeLog` compares the file size against its read offset on every poll and, when +the size comes out smaller, resets the offset to 0 and drops its partial-line buffer. + +Verified in this run: the log was truncated with `: >` while the Inspector was running, +a further tool call was driven through the shim, and the Inspector rendered it normally +as `── #7` and `── #8`. Note that the sequence numbers **kept counting** — `seq` is +scoped to the Guardian process, not to the file, so truncating the log does not restart +it at `#1`. + +## Replaying a session already recorded + +```bash +bun run inspector -- --from-start +``` + +Instead of starting at the current end, this replays everything already in the file and +then follows. Captured after the truncation above, showing the two entries the file then +held — the request's JSON body is elided at the `...` for length; everything else is +verbatim: + +``` +Envelope Inspector — tailing .acs/envelopes.jsonl (from the start) +Ctrl-C to stop. + +── #7 20:45:19.619 → REQUEST steps/toolCallRequest id=5c50a105-3981-44ac-9622-87b13fd0622c +... +── #8 20:45:20.181 ← RESPONSE steps/toolCallRequest id=5c50a105-3981-44ac-9622-87b13fd0622c +○ ALLOW +``` + +Note the `--` before the flag: `bun run inspector` is a package script, so `--` is what +separates bun's own arguments from the Inspector's. + +## The same thing through a real Claude Code session + +Wire the hook in as V1's quickstart describes: + +```bash +mkdir -p .claude +cp hosts/claude-code/settings.json .claude/settings.json +``` + +**What was actually run to write this section, stated precisely:** this was verified +through the real `claude` CLI in **headless mode** (`claude -p '' --allowedTools Bash`), +not through an interactive TUI session. The interactive session was not run, so nothing +here describes the TUI's rendering of the block. What the headless run does exercise is +the identical hook path — Claude Code loading `.claude/settings.json`, spawning +`acs-hook.ts` as a real `PreToolUse` subprocess, and honouring the returned decision. + +The payload used was `echo rm -rf /`, not `rm -rf /`. The configured pattern is matched +against the raw command string with no argv parse, so `echo rm -rf /` is denied by the +same rule — at offset 5 instead of offset 0 — while being inert if it ever did execute. +That is the payload to use for any unattended run of this demo: an unattended `rm -rf /` +is only safe for as long as the hook works, which is the thing under test. + +Captured, from terminal 2, verbatim: + +``` +── #12 20:47:42.255 ← RESPONSE steps/toolCallRequest id=f36f4b0e-0711-4673-aad0-dd7b38781f39 +● DENY reason_codes=[destructive_shell_command_blocked] policy_references=[agt_stock#destructive_shell_command_blocked] +{ + "jsonrpc": "2.0", + "id": "f36f4b0e-0711-4673-aad0-dd7b38781f39", + "result": { + "type": "final", + "acs_version": "0.1.0", + "request_id": "f36f4b0e-0711-4673-aad0-dd7b38781f39", + "decision": "deny", + "reasoning": "matched pattern (?i)rm\\s+-[a-z]*r[a-z]*f[a-z]*\\s+/(?:\\s|$) at offset 5", + "reason_codes": [ + "destructive_shell_command_blocked" + ], + "policy_references": [ + { + "policy_id": "agt_stock", + "rule_id": "destructive_shell_command_blocked" + } + ] + } +} +``` + +A second headless invocation asking for `ls -la` produced `── #13` / `── #14 ○ ALLOW`, +and the command ran normally — the CLI printed the directory listing. (It was a separate +`claude -p` invocation, so a separate `session_id`, not a second turn in the same +session.) + +**Two things the real CLI shows that the piped payload does not.** + +First, the real `tool_input` carries more than `command`. The request envelope from the +live session included a second argument the hand-written payload never has: + +``` + "arguments": { + "command": { + "value": "echo rm -rf /" + }, + "description": { + "value": "Print the literal string \"rm -rf /\"" + } + } +``` + +The hookmap maps every key of `tool_input` into `arguments`, so whatever Claude Code +sends is what appears on the wire. Watching the Inspector is the cheapest way to find +out what a host actually sends, as opposed to what its docs say it sends. + +Second, and worth knowing before you demo this live: **the agent may decline before any +hook fires.** On the first attempt the model refused to issue the Bash tool call at all, +reasoning about the payload on its own. The envelope log stayed at zero bytes — no hook, +no envelope, nothing for the Inspector to show. This is the same trap V1's runbook warns +about when it says to ask directly. Give the model the context that a governance hook is +under test and that the deny is the expected outcome, and it issues the call. If the +Inspector shows nothing at all during a live demo, check this before you check the wiring. + +## If the Inspector shows nothing + +- **The Guardian is not running, or is on another port.** The shim writes an error to + stderr and exits 1, and no envelope is ever written. V1's runbook covers this failure + in full; V2 does not change it. `N6`/`N7`, the considered fail-open/fail-closed + posture, are V3. +- **The agent never issued the tool call.** See the section above. Check the log's size: + `wc -c .acs/envelopes.jsonl`. +- **The Inspector started after the entries were written.** It starts at the current end + by default. Use `--from-start`. +- **The Guardian was constructed without a sink.** Only `packages/guardian/src/main.ts` + passes `envelopeLogPath`; a Guardian started in-process by a test does not record unless + it asks to. +- **The sink disabled itself.** A write failure disables the sink for the process lifetime + and reports once on stderr — `envelope log sink disabled after failure (): `. + It never propagates and never alters a decision: the sink is total by construction, and + the end-to-end test asserts `rm -rf /` is still denied when every sink write fails. + +## Cleaning up + +`.acs/` and `.claude/settings.json` are local artifacts. This repo never deletes +recursively: + +```bash +rm .claude/settings.json && rmdir .claude +rm .acs/envelopes.jsonl && rmdir .acs +``` diff --git a/docs/shaping/acs-reference-impl-shaping.md b/docs/shaping/acs-reference-impl-shaping.md index 8d1d1076..a995e34b 100644 --- a/docs/shaping/acs-reference-impl-shaping.md +++ b/docs/shaping/acs-reference-impl-shaping.md @@ -234,6 +234,7 @@ All resolved — see `spike-agt-integration.md`. | U22 | P4 | inspector | session chain view: SessionContext entries and lineage | render | — | — | | U23 | P4 | inspector | posture badge: negotiated `on_decision_failure`, plus a running count of audited fail-open proceeds | render | — | — | | U30 | P5 | conformance | coverage matrix, 8 intervention points × 5 verdicts | render | — | — | +| U33 | P5 | conformance | trace-pillar row: each required OTel attribute, its v0.1.0 wire source, and whether a wire consumer can emit it | render | — | — | | U31 | P5 | conformance | drift detail: changed point, verdict, or schema field | render | — | — | | U32 | P5 | conformance | rendered ACS ↔ MS-ACS mapping table | render | — | — | @@ -261,7 +262,7 @@ All resolved — see `spike-agt-integration.md`. | N23 | P3 | guardian | `assembleSnapshot()` — envelope + session state → AGT snapshot | call | → N30 | — | | N24 | P3 | guardian | `mapVerdict()` — AGT verdict → ACS decision; `warn` → `allow` + `policy_references` | call | → N25, → N26 | → N4, → N13 | | N25 | P3 | guardian | `persistResultLabels()` — AGT `result_labels` into ACS lineage | call | → S5 | — | -| N26 | P3 | guardian | `writeEnvelopeTap()` | call | → S6 | — | +| N26 | P3 | guardian | `createEnvelopeLogSink()` → `sink.write()` — ⚠️ **total**: never throws, never alters a decision. Records the request *before* validation | call | → S6 | — | | N27 | P3 | guardian | `denyOnInvalidEnvelope()` — schema or bridge failure returns an explicit ACS `deny` **decision**, not a bare error, so the host honors it instead of falling back to posture | call | → N26 | → N4, → N13 | | N28 | P3 | guardian | `buildServerHello()` — ServerHello: `timeout_config`, `on_decision_failure`, `profiles_accepted` | call | → N26 | → N5, → N14 | | N30 | P3.1 | agt-bridge | `evaluateInterventionPoint(point, snapshot)` — Node SDK | call | — | → N24 | @@ -271,6 +272,7 @@ All resolved — see `spike-agt-integration.md`. | N42 | P5 | conformance | verdict round trip: AGT verdict → ACS decision → AGT verdict, assert identity | call | — | → N47 | | N43 | P5 | conformance | `enforced_identity` recomputation check | call | — | → N47 | | N44 | P5 | conformance | failure-domain check: an AGT evaluation error arrives as an honored `deny`; a delivery failure applies the negotiated posture and writes an audit event | call | — | → N47 | +| N49 | P5 | conformance | trace-pillar check: every attribute `trace/otel-mapping.json` marks required, resolved against the v0.1.0 wire schemas — a cell is green only when a *wire consumer* could emit it | call | — | → N47 | | N45 | P5 | conformance | `fetchUpstreamSurfaces()` — AGT wire schemas and enums at `main` | call | → S12 | — | | N46 | P5 | conformance | `diffSurfaces()` — pinned versus upstream | call | — | → N47 | | N47 | P5 | conformance | `renderMatrix()` | call | → U30, → U31 | — | @@ -291,7 +293,7 @@ All resolved — see `spike-agt-integration.md`. | S3 | P3 | `sessionContext` | Hash-chained entries per `session_id` | | S4 | P3 | `intent` | Immutable Intent baseline per session | | S5 | P3 | `provenance` | `origin` / `derived_from` lineage, carrying AGT `result_labels` between steps | -| S6 | P3 | `envelope log` | JSONL of every request and response | +| S6 | P3 | `envelope log` | JSONL of every request and response as parsed, unmodified, at `.acs/envelopes.jsonl` (gitignored — carries raw tool arguments). Paired by JSON-RPC `id` | | S7 | P3.1 | `manifest.yaml` | Binds the `rego` policy to `data.agt.defaults.verdict`; declares intervention points, tools, approval | | S8 | P3.1 | `data.agt.defaults.config` | Thresholds, allowlists, pattern lists — the only place policy behaviour is authored | | S9 | P3.1 | AGT stock bundle | `policy/lib/*.rego` at the pinned ref. Requires the `opa` CLI on PATH | @@ -349,7 +351,7 @@ flowchart TB N23["N23: assembleSnapshot()"] N24["N24: mapVerdict()"] N25["N25: persistResultLabels()"] - N26["N26: writeEnvelopeTap()"] + N26["N26: createEnvelopeLogSink()"] N27["N27: denyOnInvalidEnvelope()"] N28["N28: buildServerHello()"] S3["S3: sessionContext chain"] @@ -379,11 +381,13 @@ flowchart TB U30["U30: 8x5 coverage matrix"] U31["U31: drift detail"] U32["U32: mapping table"] + U33["U33: trace-pillar row"] N40["N40: conformance runner"] N41["N41: point round trip"] N42["N42: verdict round trip"] N43["N43: enforced_identity check"] N44["N44: fail-closed check"] + N49["N49: trace-pillar check"] N45["N45: fetchUpstreamSurfaces()"] N46["N46: diffSurfaces()"] N47["N47: renderMatrix()"] @@ -480,12 +484,15 @@ flowchart TB N42 -.-> N47 N43 -.-> N47 N44 -.-> N47 + N40 --> N49 + N49 -.-> N47 N45 --> S12 S11 -.-> N46 S12 -.-> N46 N46 -.-> N47 N47 --> U30 N47 --> U31 + N47 --> U33 S10 -.-> N48 N48 --> U32 @@ -493,8 +500,8 @@ flowchart TB classDef nonui fill:#d3d3d3,stroke:#808080,color:#000 classDef store fill:#e6e6fa,stroke:#9370db,color:#000 - class U1,U2,U3,U10,U11,U12,U20,U21,U22,U23,U30,U31,U32 ui - class N1,N2,N3,N4,N5,N6,N7,N10,N11,N12,N13,N14,N15,N16,N20,N21,N22,N23,N24,N25,N26,N27,N28,N30,N31,N40,N41,N42,N43,N44,N45,N46,N47,N48,N50,N51 nonui + class U1,U2,U3,U10,U11,U12,U20,U21,U22,U23,U30,U31,U32,U33 ui + class N1,N2,N3,N4,N5,N6,N7,N10,N11,N12,N13,N14,N15,N16,N20,N21,N22,N23,N24,N25,N26,N27,N28,N30,N31,N40,N41,N42,N43,N44,N45,N46,N47,N48,N49,N50,N51 nonui class S1,S2,S3,S4,S5,S6,S7,S8,S9,S10,S11,S12,S13,S14,S15,S16 store ``` @@ -527,3 +534,16 @@ flowchart TB | ~~D7~~ | F3 — Rego or Cedar for the demo bundle | ✅ **Decided: Rego** | The deciding factor was wrong. Cedar's advantage was removing an external binary, but the SDK ships OPA 0.70.0 as a platform package — so Rego, the canonical binding, costs nothing extra. Verified: stock bundle 105/105 under the bundled OPA | | D8 | 🟡 Which `on_decision_failure` the reference ships as its default | Open, leaning `proceed` | The spec default is `proceed` (fail-open). Shipping the spec default is the honest choice, but a security-facing demo that fails open needs the audit trail on screen (U23) to read correctly. V1 negotiates and stores it (N5/N28/S13); V3 applies it (N6), so the decision is only needed by V3 | | D9 | ⚠️ **New.** Report the `./` bundle-path fail-open upstream to AGT? | Open | A `./`-prefixed `bundle:` silently voids all policy and returns `allow` with no error. It is a fail-open in a governance tool and affects any AGT host, not just us. Reporting is the good-citizen move and consistent with R4.3's non-adversarial framing; it is also unattributed outbound traffic, so it needs an explicit decision before anything is sent | +| ~~D10~~ | R5.3 — does this implementation claim the ACS **Trace** pillar? | ✅ **Decided: no, and V7 measures the non-claim** (N49 → U33). V7 does not build an exporter; per the evidence note below it could only live in the Guardian, which would be a slice of its own | `specification/v0.1.0/trace/otel-mapping.json` is normative: a deployment emitting OTel for the Trace pillar MUST use its span names and required attributes verbatim, and it maps `steps/toolCallRequest` → `gen_ai.tool.call` by name. `trace/ocsf-mapping.json` is its sibling. V2's envelope log (S6) is a raw JSONL log, deliberately not an OTel or OCSF export, so today we claim neither pillar. R5.3 requires declaring that either way. Settled: the matrix records Trace as an explicit non-claim, and **two of its required span attributes have no wire source at all — evidence note directly below** | + +**⚠️ D10 evidence — the Trace pillar is not emittable from the v0.1.0 wire alone.** Read after V2 shipped, against the pinned schemas: + +| Required by `trace/otel-mapping.json` | Source in v0.1.0 | Status | +|---|---|---| +| `gen_ai.tool.name` (on `gen_ai.tool.call`) | `payload.tool.name` — `required` | ✅ | +| `acs.capability` (on `gen_ai.tool.call`) | `payload.capability` — **optional**; `hooks/tool-call-request.json` requires only `tool` and `arguments` | ⚠️ A fully conformant envelope may omit it, so a fully conformant span cannot always be built | +| `acs.decision` (on the `acs.decision` span event) | `AcsResult.decision` — `required` | ✅ | +| `acs.evaluator` (on the `acs.decision` span event) | **none** — `AcsResult` has no `evaluator` field | ❌ No wire source at all | +| `acs.confidence`, `acs.evaluator_version`, `acs.model_id` (conditional, "required when present in the decision envelope") | **none** — no such fields in `AcsResult` | ❌ Can never be "present in the decision envelope" | + +The consequence is sharper than a missing field: a **downstream consumer of the ACS wire cannot emit a conformant trace**. Only the Guardian can, from process-local knowledge the contract does not carry. That cuts against R5.1/R5.2 — V2's whole design is that S6 is readable by anything, and the Inspector proves it by importing nothing. An OTel exporter reading S6 would hit the same wall. Either v0.2 adds `evaluator` (and friends) to `AcsResult` and promotes `capability` to required for this hook, or the Trace pillar is explicitly a Guardian-side emission, not a wire-derived one, and should say so. This is the second place this project has become a forcing function for v0.2 rather than a consumer of v0.1.0 — see the `steps/modelCall` tension under D4. diff --git a/docs/shaping/acs-reference-impl-slices.md b/docs/shaping/acs-reference-impl-slices.md index 5c3d30e3..b9743078 100644 --- a/docs/shaping/acs-reference-impl-slices.md +++ b/docs/shaping/acs-reference-impl-slices.md @@ -20,7 +20,7 @@ Every slice ends in something demo-able. | V4 | Output redaction on Claude Code | C3 | "AGT's own package documents that Claude Code cannot redact tool output. Here it is, redacted, by AGT's stock `redact` policy." | | V5 | Second host, zero AGT changes | C3 | "Same Guardian, same manifest, same bundle. OpenCode is now governed. `git diff` shows zero lines changed in the Guardian, the bridge, or AGT." | | V6 | Session state and provenance carriage | C4 | "The SessionContext chain grows per step. AGT emits `result_labels` at one step and gets them back as `input.ifc.source_labels` at the next, carried by ACS provenance." | -| V7 | Conformance matrix | C1, C2, C5 | "Eight intervention points by five verdicts, all green. AGT completely expressed in ACS, case by case." | +| V7 | Conformance matrix | C1, C2, C5 | "Eight intervention points by five verdicts, every cell resolved — green where ACS v0.1.0 expresses AGT, red with a named reason where it cannot. Plus the Trace pillar, measured as an explicit non-claim." | | V8 | Upstream drift watch | C6 | "Point the harness at AGT `main`. A changed enum turns a cell red and names the field." | **Order rationale.** V1–V4 establish credibility on the host AGT already supports best, so the second-host claim in V5 lands against a working baseline rather than a promise. V7 is the deliverable Microsoft reads, but it can only be green once V1–V6 exist to be measured. V8 is what keeps V7 true after upstream moves. @@ -57,7 +57,7 @@ Every slice ends in something demo-able. | S10 | shared | store | `mapping.yaml` | — | — | → N23, N24 | | S11 | shared | store | `agt.lock` | — | — | → N31 | -**Scope note.** Only `pre_tool_call` is wired. No session state, no tap, no second host. `N23` assembles the snapshot from the envelope alone; it starts reading S3/S4/S5 in V6. +**Scope note.** Only `pre_tool_call` is wired. No session state, no envelope log, no second host. `N23` assembles the snapshot from the envelope alone; it starts reading S3/S4/S5 in V6. **Setup cost this slice absorbs:** ⚠️ *amended* — the `opa` CLI is **no longer a setup cost*. The npm package pulls `agent-control-specification-opa-darwin-arm64`, which ships OPA 0.70.0, overridable via `ACS_OPA_PATH` / `ACS_OPA_NO_BUNDLE`. The stock bundle passes 105/105 under both it and system OPA 1.18.2. What remains: the pinned AGT checkout and the first cut of `mapping.yaml`. **This closes D7 as Rego** — Cedar's only advantage was removing an external binary, and there is no external binary. @@ -79,11 +79,33 @@ Every slice ends in something demo-able. |---|-------|-----------|------------|---------|-----------|------------| | U20 | P4 | inspector | envelope stream, request/response JSON pairs | render | — | — | | U21 | P4 | inspector | decision badge: decision + `policy_references` + `reason_codes` | render | — | — | -| N26 | P3 | guardian | `writeEnvelopeTap()` | call | → S6 | — | +| N26 | P3 | guardian | `createEnvelopeLogSink()` → `sink.write()` — **total**: never throws, never alters a decision | call | → S6 | — | | N50 | P4 | inspector | `tailEnvelopeLog()` | observe | → U20, → U21 | — | -| S6 | P3 | store | `envelope log`, JSONL | — | — | → N50 | +| S6 | P3 | store | `envelope log`, JSONL at `.acs/envelopes.jsonl` (gitignored), one entry per direction | — | — | → N50 | -**Why this early.** R5.1 and R5.2 are must-haves, and an ACS-first reader needs to see envelopes before anything more elaborate is worth showing. U21 is also how `warn` becomes visible: a `warn` arrives as `allow` with a non-empty `policy_references`, and the badge is what makes that legible rather than buried. +**Why this early.** R5.1 and R5.2 are must-haves, and an ACS-first reader needs to see envelopes before anything more elaborate is worth showing. U21 is also what makes an observe-only outcome legible rather than buried: ACS carries it as `allow` with a non-empty `policy_references` — a policy fired and the action still proceeded — and the badge is what keeps that from rendering identically to a clean allow. The upstream disposition that maps to it is deliberately not named here: ACS has no such decision, and R5.2 exists so this package carries no policy-runtime vocabulary at all — in prose as much as in identifiers, since a word in a doc teaches it as effectively as a symbol does. + +**Decisions taken at planning.** §V2 left the Inspector's form open; these close it, and are recorded here rather than only in the plan. + +| # | Decision | Rationale | +|---|---|---| +| P1 | The Inspector is a **terminal process** — `bun run inspector`, a third terminal beside `bun run guardian` and `claude`. | Zero new dependencies, works over SSH, matches the repo's one-process-per-command shape. R7.1/R7.2 ask for a laptop and no paid dependency; a browser UI would add a server, a bundler and an asset pipeline without proving anything further about the wire. A browser view later reads the same S6 file. | +| P2 | S6 is a **file** — `.acs/envelopes.jsonl`, overridable with `ACS_ENVELOPE_LOG`. | The file is the seam that lets the Inspector import nothing from the Guardian. `jq` works on it unchanged. An in-process bus or a socket would couple P4 to P3. | +| P3 | The sink is **opt-in at the library level, on by default in the CLI**: `startGuardian` records only when `envelopeLogPath` is passed; `packages/guardian/src/main.ts` passes it. | V1's tests construct Guardians constantly; a default-on sink would scatter files through the working tree. The demo path still gets the log with nobody opting in. | +| P4 | Request/response pairing is by **JSON-RPC `id`**, carried as `rpc_id` on every entry. | The only identifier present in both directions. `params.request_id` exists on requests only. Pairing by arrival order breaks the moment two hooks are in flight. | +| P5 | The request is recorded **before validation**. | An envelope that fails the schema is the most useful thing an ACS-first reader can see, and it is exactly what disappears if the sink sits behind the validator. R5.1 says *every* hook firing. | + +**⚠️ Watch-for — the envelope log sink must be total.** `sink.write()` sits on the decision path. V1 shipped three separate fail-opens before they were caught (the `./` bundle landmine, `tool_unknown` failing closed, and an unhandled Guardian throw reaching the shim as an empty stdout); an observability feature that can turn a governed tool call into an ungoverned one would be the fourth. Every write is wrapped: a failure disables the sink for the process lifetime, reports once, and never propagates. V2 asserts this end to end — `rm -rf /` is still denied when every sink write fails. + +**⚠️ Watch-for — S6 records the parsed envelope, unmodified.** No field stripping, no redaction, no reordering of anything we control; pretty-printing happens at render time only. An inspector that shows something other than what was sent is worse than none. The consequence is that S6 carries raw tool arguments, which is why `.acs/` is gitignored and why the runbook says so out loud. **Corrected by V2's whole-branch review:** this watch-for originally said "records the wire verbatim", and so did the plan's global constraint 11, the slice README, the runbook, and the Inspector's own renderer comment. The sink is handed `await req.json()`, so it stores a JSON *value*, not bytes — the parse collapses duplicate keys, canonicalises number literals, and hoists integer-like object keys, and `arguments` keys are host-controlled. Storing bytes instead would make `envelope` a string rather than JSON, costing the Inspector its pretty-printing and the round-trip contract test its subject. The wording was corrected everywhere rather than the code. + +**⚠️ Watch-for — a schema failure appears as an error, not a decision.** In V2 an invalid envelope is recorded (P5) and then answered with a JSON-RPC error, so the Inspector renders `✖ ERROR -32010`, not a badge. `N27 denyOnInvalidEnvelope()` — the affordance that turns Guardian-side failures into honoured ACS `deny` **decisions** — is V3. The Inspector is where that change will become visible. + +**Unpaired responses are real.** A body that will not parse as JSON produces a response with no preceding request and `rpc_id: null`. The Inspector renders it as `(no method) (unpaired)` rather than hiding it. + +**Scope added at planning** (both amend this slice, both land in V2's PR): +- An **invariant gate** on `packages/inspector/src`: zero AGT vocabulary, zero host vocabulary, and no import of `guardian` or `agt-bridge`. R5.2 is why this slice is early, and V1 established that this project turns architectural claims into grep gates rather than prose. Joins the R3.2/R3.3 gates in `test/invariants.test.ts`. +- A **write↔tail contract test** (`test/envelope-log-sink-roundtrip.test.ts`). The Inspector declares its own `EnvelopeLogEntry` instead of importing the Guardian's — that is what makes the gate above meaningful — and the duplication is only safe while something fails when the two drift. --- @@ -173,25 +195,43 @@ Wire N21 → N22 → N23 in place of V1's direct N21 → N23. ## V7: Conformance matrix -**Demo:** Eight intervention points by five verdicts, all green. AGT completely expressed in ACS, case by case. +**Demo:** Eight intervention points by five verdicts, every cell resolved — green where ACS v0.1.0 expresses AGT, red with a named reason where it cannot. Plus the Trace pillar, measured as an explicit non-claim. + +**⚠️ Demo corrected (was "all green").** The original sentence was already contradicted by this slice's own body, which has expected two honestly-red model-call cells since shaping; D10 adds two more. A matrix that must be all green to count is a matrix under pressure to redefine the claim, which is the opposite of what C2 is for. The demo now asks for every cell *resolved*, which is achievable and is the stronger deliverable. | # | Place | Component | Affordance | Control | Wires Out | Returns To | |---|-------|-----------|------------|---------|-----------|------------| | U30 | P5 | conformance | coverage matrix, 8 intervention points × 5 verdicts | render | — | — | | U32 | P5 | conformance | rendered ACS ↔ MS-ACS mapping table | render | — | — | +| U33 | P5 | conformance | trace-pillar row: each required OTel attribute, its v0.1.0 wire source, and whether a wire consumer can emit it | render | — | — | | N40 | P5 | conformance | `acs-agt-conformance` runner | call | → N41, → N42, → N43, → N44 | — | | N41 | P5 | conformance | intervention-point round trip, validated against `policy-input.schema.json` | call | — | → N47 | | N42 | P5 | conformance | verdict round trip: AGT verdict → ACS decision → AGT verdict, assert identity | call | — | → N47 | | N43 | P5 | conformance | `enforced_identity` recomputation check | call | — | → N47 | | N44 | P5 | conformance | failure-domain check: an AGT evaluation error arrives as an honored `deny`; a delivery failure applies the negotiated posture and writes an audit event | call | — | → N47 | -| N47 | P5 | conformance | `renderMatrix()` | call | → U30 | — | +| N49 | P5 | conformance | trace-pillar check: every attribute `trace/otel-mapping.json` marks required, resolved against the v0.1.0 wire schemas | call | — | → N47 | +| N47 | P5 | conformance | `renderMatrix()` | call | → U30, → U33 | — | | N48 | P5 | conformance | `renderMappingTable()` | call | → U32 | — | **⚠️ Gap discovered in V1 — the Guardian's outbound envelopes are validated by nothing.** Inbound requests get Ajv against all 43 v0.1.0 schemas (N21), but responses are hand-built objects checked by no schema. The conformance harness would therefore measure a wire format that was never itself contract-checked — which quietly weakens exactly the claim C2 exists to prove. Add response validation before the matrix is published. Related: V1 found that `response-envelope.json`'s `result` unconditionally `$ref`s `AcsResult`, which requires `decision` — a ServerHello has no such field, so a handshake response cannot satisfy it. That looks like a genuine v0.1.0 spec gap (no discriminated union for non-decision methods) and is worth an upstream ACS issue, not just a red cell. -**Expect two cells to be honestly red.** `pre_model_call` and `post_model_call` have no ACS v0.1.0 target — see D4. Red cells with a stated reason are worth more than a green matrix that quietly redefines the claim, and they are the forcing function for `steps/modelCall` in v0.2. +**⚠️ The Trace pillar lands here too (D10), and two of its cells are already known red.** `trace/otel-mapping.json` is normative — a deployment emitting OTel for the Trace pillar MUST use its span names and required attributes verbatim. Measured against the pinned schemas after V2 shipped: + +| Required by the mapping | Source in v0.1.0 | Cell | +|---|---|---| +| `gen_ai.tool.name` on `gen_ai.tool.call` | `payload.tool.name`, `required` | 🟢 | +| `acs.capability` on `gen_ai.tool.call` | `payload.capability`, **optional** — `hooks/tool-call-request.json` requires only `tool` and `arguments` | 🔴 a conformant envelope may omit it | +| `acs.decision` on the `acs.decision` span event | `AcsResult.decision`, `required` | 🟢 | +| `acs.evaluator` on the `acs.decision` span event | **none** — `AcsResult` has no such field | 🔴 no wire source | +| `acs.confidence`, `acs.evaluator_version`, `acs.model_id` (required "when present in the decision envelope") | **none** — no such fields in `AcsResult` | 🔴 can never be present | + +`N49` is what turns that table into measured cells rather than this prose, and `U33` renders it. The finding worth publishing is not the missing fields but their consequence: **a downstream consumer of the ACS wire cannot emit a conformant trace** — only the Guardian can, from process-local knowledge the contract does not carry. That cuts directly against R5.1/R5.2 and against V2's design, where S6 is readable by anything and the Inspector proves it by importing nothing. An OTel exporter reading S6 hits the same wall. + +**Scope boundary:** V7 *measures* the Trace pillar. It does not build an exporter. If an exporter is ever wanted it has to live in the Guardian for the reason above, and that is a slice of its own, not V7 scope. -R5.3 lands here: the matrix *is* the profile declaration. +**Expect two cells to be honestly red — now four.** `pre_model_call` and `post_model_call` have no ACS v0.1.0 target (D4), and the two Trace attributes above have no wire source (D10). Red cells with a stated reason are worth more than a green matrix that quietly redefines the claim, and they are the forcing function for `steps/modelCall` and for an `evaluator` field on `AcsResult` in v0.2. + +R5.3 lands here: the matrix *is* the profile declaration — including the Trace pillar, which this implementation declares it does **not** claim, with the measured reason attached. --- @@ -224,6 +264,8 @@ Runs on a schedule in CI. MS-ACS is `0.3.1-beta` and warns of breaking changes b | 6 | ⚠️ A `./`-prefixed `bundle:` path silently disables policy — every decision becomes `allow`, with no error | V1 | `createBridge` throws on `/./`; V1's deny test is the backstop. Worth reporting upstream: a fail-open in a governance tool | | 7 | ⚠️ `enforced_identity` bisection is unavailable over AGT's Python binding | V7 | Resolved by embedding the **Node** SDK, which serializes `input_identity` and `enforced_identity` distinctly. Had we stayed on Python, R1.4 would be unverifiable and N43 impossible | | 8 | ⚠️ AGT's verdict carries no `rule_id` / `reason_codes` / `reasoning` | V1, V7 | `mapVerdict` synthesizes them from `reason` / `message`, and `mapping.yaml` is where that synthesis is declared — so V7 measures it rather than assuming it | +| 9 | ⚠️ S6 grows unbounded — no rotation and no size cap | V2 | Accepted. It is a gitignored local demo artifact; `: > .acs/envelopes.jsonl` truncates it safely mid-run because `tailEnvelopeLog` resets on truncation. Rotation is not built, and the runbook says so | +| 10 | ⚠️ The sink's two synchronous `appendFileSync` calls per request sit **on the decision path**, and `Bun.serve` is single-threaded | V2 | Accepted, and correct for demo scale. Surfaced by V2's whole-branch review as the neighbour of row 9: a slow filesystem (a stalled network mount, a full disk) blocks *every* in-flight request, not only the one being recorded, because there is no second thread to run them on. No correctness risk — the sink is total, so a write that fails degrades observability and never a decision (constraint 8) — and no latency budget is claimed for it. Recorded rather than fixed; an async or queued sink is the change if a deployment ever needs one | ## Open decisions carried from shaping @@ -235,5 +277,10 @@ Runs on a schedule in CI. MS-ACS is `0.3.1-beta` and warns of breaking changes b | D5 | Determinism of the demo | V1 onward | | ~~D7~~ | ✅ **Closed: Rego.** Cedar's sole advantage was avoiding an external binary; the SDK bundles OPA, so that advantage does not exist. Stock bundle verified 105/105 under the bundled OPA | ~~V1~~ | | D8 | 🟡 Which `on_decision_failure` ships as default — V1 negotiates and stores it (N5/N28/S13); V3 applies it (N6). Leaning to the spec default `proceed`, paired with U23's audit count | V3 | +| ~~D10~~ | ✅ **Closed: V7 owns it, as a measured non-claim.** The ACS Trace pillar (`trace/otel-mapping.json`, `trace/ocsf-mapping.json`) is normative and was unclaimed by any slice. It now lands in V7 as `N49`/`U33` — V7 *measures* the pillar rather than emitting it, and the matrix declares it as a pillar this implementation does not claim, with the reason attached. Two required attributes already measure red: `acs.evaluator` has no field in `AcsResult` at all, and `acs.capability` maps to an optional payload field. The consequence is the publishable part — **a downstream consumer of the ACS wire cannot emit a conformant trace**, only the Guardian can, from knowledge the contract does not carry. Building an exporter would be a slice of its own, not V7 scope. Evidence tables in §V7 and in the shaping doc under D10 | `specification/v0.1.0/trace/otel-mapping.json` and `trace/ocsf-mapping.json` are *normative* — the OTel mapping states that a deployment emitting OTel for the Trace pillar MUST use its span names and required attributes verbatim, and it names `steps/toolCallRequest` → `gen_ai.tool.call` explicitly. V2's S6 is deliberately a raw envelope log, **not** an OTel or OCSF export, so this implementation currently claims neither. R5.3 says we declare what we claim and what we do not. Surfaced during V2 planning, measured after V2 shipped, and settled into V7 | V7 (N49, U33) | **Correction log.** V1 planning verified the AGT surface by running it rather than reading it, and produced ten corrections — the SDK choice, the `./` landmine, config-inside-the-bundle, the absent stock shell patterns, the leaf `policy_target`, AGT's missing `rule_id`/`reason_codes`/`reasoning`, lowercase wire decisions, `steps/toolCallRequest` and the 19-hook count, the retired `opa` setup cost, and the Python identity collapse. Each is recorded above at the row it governs, with its evidence, in `docs/superpowers/plans/2026-08-09-v1-one-host-one-hook.md`. + +V2 planning produced no corrections — §V2 had nothing wrong in it — but it did close five open choices (P1–P5, recorded under §V2), add three watch-fors, add risk row 9, and surface D10. Its plan is `docs/superpowers/plans/2026-08-09-v2-envelope-inspector.md`. + +**V2's whole-branch review produced one correction of its own**, recorded at the watch-for it governs: "S6 records the wire verbatim" over-claimed byte identity that the implementation never had, and the over-claim had propagated verbatim from the plan's global constraint 11 into the slice README, the runbook, the shaping doc's S6 row, and the Inspector's renderer. Corrected in wording, not in code — see the watch-for above for why storing raw bytes would be the worse trade. The same review added risk row 10. diff --git a/docs/superpowers/plans/2026-08-09-v2-envelope-inspector.md b/docs/superpowers/plans/2026-08-09-v2-envelope-inspector.md new file mode 100644 index 00000000..8272532a --- /dev/null +++ b/docs/superpowers/plans/2026-08-09-v2-envelope-inspector.md @@ -0,0 +1,1831 @@ +# V2 — Envelope Inspector + +## Slice Contract + +| Field | Value | +|---|---| +| Slice ID | [#3](https://github.com/afogel/ACS_reference_implementation/issues/3) (epic [#1](https://github.com/afogel/ACS_reference_implementation/issues/1), PR [#11](https://github.com/afogel/ACS_reference_implementation/pull/11), stacked on `slice/v1`) | +| Slices doc | `docs/shaping/acs-reference-impl-slices.md` §V2, line 74 | +| Demo | "Watch the ACS request and response JSON stream live while you work in Claude Code." | +| Components | U20 envelope stream (request/response JSON pairs) · U21 decision badge (decision + `policy_references` + `reason_codes`) · N26 `writeEnvelopeTap()` · N50 `tailEnvelopeLog()` · S6 envelope log, JSONL | +| Parked items | U22 session chain view → **V6** (needs S3/S4/S5, which do not exist yet). U23 posture badge and N51 `tailAuditSinks()` → **V3** (need S14, the audit sink). N27 `denyOnInvalidEnvelope()` → **V3**. | +| Watch-for | None recorded in §V2. Three are **added by this plan** (tap totality, pre-validation request tap, unpaired responses) and amended into the slices doc in this PR. | +| Corrections | None marked in §V2. | +| Requirements | R5.1 (every hook firing inspectable as an ACS envelope), R5.2 (an ACS-first reader traces one action end to end without reading AGT source), R7.1, R7.2 | + +**Global Constraints** (bind every task; copy verbatim into reviewer dispatches): + +1. **Zero Rego authored.** Policy behaviour is configured only through `data.agt.defaults.config`. No `.rego` file is written or edited by us. (R2.1) +2. **Stock bundle byte-identical.** Every `.rego` under `policy/lib/` matches AGT at the pinned ref exactly. The only permitted addition to that directory is `data.json`. (R2.2, R2.3) +3. **The host adapter contains zero AGT-specific code.** No file under `packages/host-adapter/` may mention AGT, Rego, OPA, verdicts, or intervention points. (R3.2) +4. **The AGT bridge contains zero host-specific code.** No file under `packages/agt-bridge/` may mention Claude Code, hooks, OpenCode, or stdin/stdout hook protocols. (R3.3) +5. **ACS decisions are lowercase on the wire** — `allow`, `deny`, `modify`, `ask`, `defer`. Uppercase appears in spec prose and in *rendered output* only, never on the wire. +6. **The bundle path in `policy/manifest.yaml` must not begin with `./`.** A `./` prefix silently disables the entire policy and every decision becomes `allow`. +7. **AGT is stateless.** Nothing under `packages/agt-bridge/` persists anything between calls. (R6.1) +8. **The tap is total.** `writeEnvelopeTap` must never throw, and a tap failure must never change, delay, or suppress a decision. Observability degrades; governance does not. V1 shipped three separate fail-opens before they were caught — the tap is new code on the decision path and gets this constraint explicitly. +9. **The Inspector contains zero AGT vocabulary and zero host vocabulary.** It reads ACS envelopes as data and knows nothing about AGT or about Claude Code. That is R5.2 stated as a property of the code, and Task 6 enforces it by grep. +10. **The Inspector imports nothing from `guardian` or `agt-bridge`.** It reads the log file. A compile-time dependency would make "inspectable on the wire" (R5.1) a claim about our own type graph rather than about the wire. +11. **S6 records the JSON value the Guardian parsed, unmodified** — no field stripping, no redaction, no reordering of anything we control. Pretty-printing happens at render time only. An inspector that shows something other than what was sent is worse than no inspector. + + > **Amended after V2's whole-branch review (finding 2).** This constraint originally read "S6 records exactly what crossed the wire", and that was never true of the design it governed: the tap is handed `await req.json()`, so it stores a JSON *value*, not the request's bytes. The parse collapses duplicate keys, canonicalises number literals (`1.0` → `1`, `1e2` → `100`), and hoists integer-like object keys ahead of the rest — and tool argument names are host-controlled, so an `arguments` object carrying `"0"` alongside `"a"` is a real shape, not a hypothetical. + > + > The over-promise propagated verbatim out of this line into `packages/inspector/src/render.ts`, `docs/demos/v2-runbook.md`, `slices/v2/README.md`, the shaping doc's S6 affordance row, the slices doc's V2 watch-for, and two test titles — every one of them corrected in the same wave, in wording rather than in code. Storing raw bytes would make the entry's `envelope` field a string rather than a JSON value, which breaks the Inspector's pretty-printing and the round-trip contract test: a worse trade than an accurate sentence. The code snippets embedded further down this plan still show the original wording; they are the historical record of what was planned, and this note is what corrects them. +12. **No new runtime dependencies.** Bun and TypeScript `strict` + `noUncheckedIndexedAccess` only, as in V1. + +--- + +## Decisions taken during planning + +These are choices §V2 leaves open. Each is amended into the slices doc in this PR. + +| # | Decision | Why | What it rules out | +|---|---|---|---| +| P1 | **The Inspector is a terminal process** — `bun run inspector`, a third terminal beside `bun run guardian` and `claude`. | Zero new dependencies (constraint 12), works over SSH, and matches the repo's existing one-process-per-command shape. R7.1/R7.2 are about a laptop and no paid dependency; a browser UI would add a server, a bundler, and an asset pipeline without proving anything further about the wire. | A web UI, SSE, WebSockets, and any bundler. If a browser view is wanted later it reads the same S6 file and costs nothing already built here. | +| P2 | **S6 is `.acs/envelopes.jsonl`**, gitignored, one JSON object per line. Overridable with `ACS_ENVELOPE_LOG`. | A file is the seam that makes constraint 10 possible: the Guardian writes, anything at all reads. `jq` works on it unchanged. | An in-process event bus or a socket between Guardian and Inspector — either would couple P3 to P4. | +| P3 | **The Guardian's tap is opt-in at the library level, on by default in the CLI.** `startGuardian` taps only when `envelopeLogPath` is passed; `packages/guardian/src/main.ts` passes it. | Existing V1 tests construct Guardians constantly; a default-on tap would scatter files through the working tree and make test output order-dependent. The demo path still gets the tap without anyone opting in. | A module-level singleton tap, or a default path baked into `startGuardian`. | +| P4 | **Request/response pairing is by JSON-RPC `id`**, carried on every entry as `rpc_id`. | It is the only identifier present on both directions of a JSON-RPC exchange. `params.request_id` exists on requests only. | Pairing by arrival order, which breaks the moment two hooks are in flight. | +| P5 | **The request is tapped *before* validation.** | An envelope that fails schema validation is the single most useful thing an ACS-first reader can see, and it is exactly what disappears if the tap sits after the validator. R5.1 says *every* hook firing. | Tapping only well-formed envelopes. | + +--- + +## Slice accounting + +| From the slice | Handled by | Note | +|---|---|---| +| U20 envelope stream, request/response JSON pairs | Task 4 (`renderEntry`), Task 5 (CLI) | | +| U21 decision badge: decision + `policy_references` + `reason_codes` | Task 4 (`renderDecisionBadge`) | Includes the `warn`-as-`allow`-with-`policy_references` case the slice calls out by name | +| N26 `writeEnvelopeTap()` | Task 1 (module), Task 2 (wiring) | | +| N50 `tailEnvelopeLog()` | Task 3 | | +| S6 envelope log, JSONL | Task 1 (format), Task 2 (path + gitignore) | | +| "Why this early": R5.1 | Tasks 1, 2 | Every firing reaches S6, including envelopes that fail validation | +| "Why this early": R5.2 | Task 6 | Enforced as a grep gate, not asserted in prose | +| "Why this early": `warn` becomes visible through U21 | Task 4 | A dedicated badge state, with its own test | +| Parked → V3: U23 posture badge, N51 `tailAuditSinks()`, N27 | not in this plan | stays V3's | +| Parked → V6: U22 session chain view | not in this plan | stays V6's | +| R7.1 one command on a laptop | Task 6 | Runbook; `bun install` adds nothing | +| R7.2 no paid dependency | Task 6 | Constraint 12 makes it structural | + +--- + +## Cross-slice work in this plan + +| Task | Belongs to | Why it must happen here | +|---|---|---| +| Task 2's refactor of `handleAcsRequest` into parse → tap → `dispatch` → tap | V1's N20 | The tap has to see every response including the JSON-RPC parse error, and V1 left six `return` sites inside one function. Tapping at six sites would guarantee the seventh is missed when V3 adds N27. This is the smallest change that makes the tap total by construction. | + +## Scope added during planning + +| What | Why the slice cannot ship without it | Slices-doc amendment | +|---|---|---| +| An invariant gate on `packages/inspector/src` (zero AGT vocabulary, zero host vocabulary, no import of `guardian`/`agt-bridge`) | R5.2 is the reason §V2 says "why this early". V1 established that this project turns architectural claims into grep gates rather than prose; an Inspector that merely *happens* not to import AGT proves nothing next slice. | New line under §V2 recording the gate | +| A cross-package round-trip test (Guardian tap writes → Inspector tail reads → badge renders) | The Inspector declares its own `TapEntry` rather than importing the Guardian's (constraint 10). That duplication is only safe if something fails when the two drift. | New line under §V2 recording the contract test | +| `.gitignore` entry for `.acs/` | S6 lands in the working tree the first time the demo runs, and it contains raw tool arguments. | Recorded in the S6 amendment | +| Risk row: S6 grows unbounded | Honest limitation of a JSONL tap with no rotation. | New risk row 9 | +| Open decision D10: the ACS Trace pillar is unclaimed | `spec/acs/specification/v0.1.0/trace/otel-mapping.json` and `trace/ocsf-mapping.json` are normative mappings no slice claims. V2's tap is deliberately *not* an OTel/OCSF export, and R5.3 says this implementation declares what it claims and what it does not. Recording it is the honest move; building it is not this slice. | New D10 row | + +--- + +## Tasks + +### Task 1: S6's format and N26 `writeEnvelopeTap()` · slice #3 · N26, S6 + +**Files:** +- Create: `packages/guardian/src/envelope-tap.ts` +- Test: `packages/guardian/test/envelope-tap.test.ts` + +**Interfaces:** +- Produces: `createEnvelopeTap(options): EnvelopeTap`, `NULL_TAP: EnvelopeTap`, `extractRpcId(envelope): string | number | null`, and the types `TapEntry`, `TapDirection`, `EnvelopeTap`, `CreateEnvelopeTapOptions`. Task 2 wires `createEnvelopeTap` and `NULL_TAP` into `server.ts`. Task 3 re-declares `TapEntry` independently in the Inspector (constraint 10); Task 5's round-trip test is what keeps the two honest. + +- [ ] **Step 1: Write the failing test** — `packages/guardian/test/envelope-tap.test.ts` + +```ts +import { describe, expect, it } from "bun:test"; +import { mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { createEnvelopeTap, extractRpcId, NULL_TAP, type TapEntry } from "../src/envelope-tap.ts"; + +/** A temp directory per test. Cleanup is deliberately non-recursive -- + * unlink the one file we created, then rmdir -- so a stray file makes the + * test fail loudly instead of being silently blown away. */ +function withTempDir(run: (dir: string) => void): void { + const dir = mkdtempSync(join(tmpdir(), "acs-tap-")); + try { + run(dir); + } finally { + try { + unlinkSync(join(dir, "envelopes.jsonl")); + } catch { + // the test may not have produced a log at all -- that is the point of some of them + } + rmdirSync(dir); + } +} + +function readEntries(path: string): TapEntry[] { + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as TapEntry); +} + +const REQUEST = { jsonrpc: "2.0", method: "steps/toolCallRequest", id: 7, params: { acs_version: "0.1.0" } }; +const RESPONSE = { jsonrpc: "2.0", id: 7, result: { decision: "deny" } }; + +describe("createEnvelopeTap (N26) -- S6's JSONL format", () => { + it("writes one line per call, with a monotonic seq starting at 1", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const tap = createEnvelopeTap({ path }); + + tap.write("request", REQUEST, "steps/toolCallRequest"); + tap.write("response", RESPONSE, "steps/toolCallRequest"); + + const entries = readEntries(path); + expect(entries.map((e) => e.seq)).toEqual([1, 2]); + expect(entries.map((e) => e.direction)).toEqual(["request", "response"]); + }); + }); + + it("records the envelope verbatim -- constraint 11, no reformatting or stripping", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + createEnvelopeTap({ path }).write("request", REQUEST, "steps/toolCallRequest"); + + expect(readEntries(path)[0]?.envelope).toEqual(REQUEST); + }); + }); + + it("carries rpc_id from both directions, so the Inspector can pair them (P4)", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const tap = createEnvelopeTap({ path }); + + tap.write("request", REQUEST, "steps/toolCallRequest"); + tap.write("response", RESPONSE, "steps/toolCallRequest"); + + expect(readEntries(path).map((e) => e.rpc_id)).toEqual([7, 7]); + }); + }); + + it("stamps recorded_at from the injected clock", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const tap = createEnvelopeTap({ path, now: () => new Date("2026-08-09T12:04:31.221Z") }); + + tap.write("request", REQUEST, "steps/toolCallRequest"); + + expect(readEntries(path)[0]?.recorded_at).toBe("2026-08-09T12:04:31.221Z"); + }); + }); + + it("records method as null when the caller cannot determine one", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + createEnvelopeTap({ path }).write("response", { jsonrpc: "2.0", id: null, error: { code: -32700 } }, null); + + const entry = readEntries(path)[0]; + expect(entry?.method).toBeNull(); + expect(entry?.rpc_id).toBeNull(); + }); + }); + + // Global constraint 8. This is the whole reason the tap is a module and + // not three inline appendFileSync calls. + it("never throws when the log path is unwritable, reports once, and goes quiet", () => { + withTempDir((dir) => { + const blocker = join(dir, "envelopes.jsonl"); + writeFileSync(blocker, ""); + // A path *through* a regular file: mkdirSync and appendFileSync both + // fail with ENOTDIR, deterministically, on every platform. + const path = join(blocker, "nested", "envelopes.jsonl"); + const errors: unknown[] = []; + const tap = createEnvelopeTap({ path, onError: (error) => errors.push(error) }); + + expect(() => tap.write("request", REQUEST, "steps/toolCallRequest")).not.toThrow(); + expect(() => tap.write("response", RESPONSE, "steps/toolCallRequest")).not.toThrow(); + expect(errors.length).toBe(1); + }); + }); + + it("never throws on an envelope JSON.stringify cannot serialize", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const errors: unknown[] = []; + const tap = createEnvelopeTap({ path, onError: (error) => errors.push(error) }); + const circular: Record = { id: 1 }; + circular.self = circular; + + expect(() => tap.write("request", circular, "steps/toolCallRequest")).not.toThrow(); + expect(errors.length).toBe(1); + }); + }); + + it("NULL_TAP writes nothing and never throws", () => { + expect(() => NULL_TAP.write("request", REQUEST, "steps/toolCallRequest")).not.toThrow(); + expect(NULL_TAP.path).toBeNull(); + }); +}); + +describe("extractRpcId", () => { + it("reads string and number ids", () => { + expect(extractRpcId({ id: 7 })).toBe(7); + expect(extractRpcId({ id: "abc" })).toBe("abc"); + }); + + it("returns null for a missing, null, or non-scalar id", () => { + expect(extractRpcId({})).toBeNull(); + expect(extractRpcId({ id: null })).toBeNull(); + expect(extractRpcId({ id: { nested: true } })).toBeNull(); + expect(extractRpcId("not an object")).toBeNull(); + }); +}); +``` + +- [ ] **Step 2: Run it, expect FAIL** + +```bash +bun test packages/guardian/test/envelope-tap.test.ts +``` + +Expected: every test errors on the unresolved import of `../src/envelope-tap.ts`. + +- [ ] **Step 3: Minimal implementation** — `packages/guardian/src/envelope-tap.ts` + +```ts +/** + * writeEnvelopeTap (N26) writes S6: a JSONL record of every ACS envelope + * that crosses this Guardian's wire, in both directions, exactly as it + * crossed (global constraint 11). The Envelope Inspector (P4) reads this + * file and nothing else -- see packages/inspector, which deliberately + * imports nothing from here. + * + * Total by construction (global constraint 8). Every write is wrapped: a + * failure disables the tap for the process lifetime, reports once, and is + * never propagated to the caller. The tap sits on the decision path, and + * V1 shipped three separate fail-opens before they were caught -- an + * observability feature that can turn a governed tool call into an + * ungoverned one would be the fourth. Observability degrades; governance + * does not. + */ +import { appendFileSync, mkdirSync } from "node:fs"; +import { dirname } from "node:path"; + +export type TapDirection = "request" | "response"; + +/** + * One line of S6. `envelope` is the JSON-RPC object verbatim -- request or + * response -- and every other field is Guardian-side context the wire does + * not carry: a sequence number so a reader can detect gaps, a timestamp, the + * direction, the ACS method (JSON-RPC responses carry none, so the Guardian + * supplies the one it dispatched), and the JSON-RPC id that pairs the two + * directions. + */ +export type TapEntry = { + seq: number; + recorded_at: string; + direction: TapDirection; + method: string | null; + rpc_id: string | number | null; + envelope: unknown; +}; + +export type EnvelopeTap = { + write(direction: TapDirection, envelope: unknown, method: string | null): void; + readonly path: string | null; +}; + +export type CreateEnvelopeTapOptions = { + path: string; + /** Injectable clock, so tests can assert recorded_at exactly. */ + now?: () => Date; + /** Called at most once, on the first failure. Defaults to one stderr line. */ + onError?: (error: unknown) => void; +}; + +/** The tap a Guardian gets when no envelopeLogPath was configured (P3). */ +export const NULL_TAP: EnvelopeTap = { + path: null, + write(): void {}, +}; + +/** The JSON-RPC id, when it is a scalar. Both request and response envelopes + * carry `id` at the top level, so one extractor serves both directions. */ +export function extractRpcId(envelope: unknown): string | number | null { + if (typeof envelope === "object" && envelope !== null && "id" in envelope) { + const id = (envelope as { id: unknown }).id; + if (typeof id === "string" || typeof id === "number") { + return id; + } + } + return null; +} + +export function createEnvelopeTap({ path, now = () => new Date(), onError }: CreateEnvelopeTapOptions): EnvelopeTap { + let seq = 0; + let disabled = false; + + const fail = (error: unknown): void => { + disabled = true; + if (onError) { + onError(error); + return; + } + const message = error instanceof Error ? error.message : String(error); + console.error(`envelope tap disabled after failure (${path}): ${message}`); + }; + + try { + mkdirSync(dirname(path), { recursive: true }); + } catch (error) { + fail(error); + } + + return { + path, + write(direction, envelope, method): void { + if (disabled) { + return; + } + try { + const entry: TapEntry = { + seq: seq + 1, + recorded_at: now().toISOString(), + direction, + method, + rpc_id: extractRpcId(envelope), + envelope, + }; + const line = `${JSON.stringify(entry)}\n`; + appendFileSync(path, line); + seq += 1; + } catch (error) { + fail(error); + } + }, + }; +} +``` + +- [ ] **Step 4: Run it, expect PASS** + +```bash +bun test packages/guardian/test/envelope-tap.test.ts +bun run typecheck +``` + +- [ ] **Step 5: Commit** + +``` +Add writeEnvelopeTap and S6's JSONL entry format (N26, S6) + +Slice: #3 +Affordances: N26, S6 +``` + +--- + +### Task 2: Wire the tap into the Guardian · slice #3 · N26, N20 + +**Files:** +- Modify: `packages/guardian/src/server.ts` (lines 85-173 — `startGuardian` and `handleAcsRequest`) +- Modify: `packages/guardian/src/index.ts` (append exports) +- Modify: `packages/guardian/src/main.ts` (pass the log path) +- Modify: `.gitignore` (append) +- Test: `packages/guardian/test/envelope-tap-wiring.test.ts` + +**Interfaces:** +- Consumes: `createEnvelopeTap`, `NULL_TAP`, `EnvelopeTap` from `./envelope-tap.ts` (Task 1). +- Produces: `StartGuardianOptions.envelopeLogPath?: string`. Task 5's round-trip test passes it. + +**Requirements:** + +`handleAcsRequest` currently has six `return` sites. Tapping at each is how the seventh gets missed. Restructure it into exactly three phases, so totality is structural rather than remembered: + +``` +parse → (on failure: tap the error response, return) +tap the request → dispatch() → tap the response → return +``` + +`dispatch()` holds V1's existing logic verbatim — validate, handshake branch, toolCallRequest branch, not-dispatched branch — and returns a response instead of the caller returning it directly. Do not change any decision, any error code, or any message text: this task adds a tap and moves code, nothing else. V1's `packages/guardian/test/server.test.ts` must pass untouched. + +Add a best-effort `extractMethod(raw): string | null` beside the existing `extractId`, used only to label tap entries. + +- [ ] **Step 1: Write the failing test** — `packages/guardian/test/envelope-tap-wiring.test.ts` + +```ts +import { describe, expect, it } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startGuardian } from "../src/index.ts"; +import type { TapEntry } from "../src/envelope-tap.ts"; + +function makeEnvelope( + method: string, + payload: Record, + overrides: { id?: number; requestId?: string } = {}, +): Record { + const { id = 1, requestId = crypto.randomUUID() } = overrides; + return { + jsonrpc: "2.0", + method, + id, + params: { + acs_version: "0.1.0", + request_id: requestId, + timestamp: new Date().toISOString(), + metadata: { agent_id: "agent-1", session_id: crypto.randomUUID() }, + payload, + }, + }; +} + +function toolCallEnvelope(command: string, overrides: { id?: number } = {}) { + return makeEnvelope( + "steps/toolCallRequest", + { tool: { name: "run_shell" }, arguments: { command: { value: command } } }, + overrides, + ); +} + +function readEntries(path: string): TapEntry[] { + if (!existsSync(path)) { + return []; + } + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as TapEntry); +} + +/** Non-recursive cleanup, as in envelope-tap.test.ts. */ +async function withGuardian( + logPathFor: (dir: string) => string, + run: (url: string, logPath: string) => Promise, +): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-tap-wiring-")); + const logPath = logPathFor(dir); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath }); + try { + await run(guardian.url, logPath); + } finally { + await guardian.close(); + try { + unlinkSync(logPath); + } catch { + // some tests deliberately make the path unwritable + } + try { + rmdirSync(dir); + } catch { + // a blocker file may remain; the assertions already covered what matters + } + } +} + +async function postRaw(url: string, body: string): Promise { + const res = await fetch(url, { method: "POST", headers: { "content-type": "application/json" }, body }); + return await res.json(); +} + +const logIn = (dir: string) => join(dir, "envelopes.jsonl"); + +describe("Guardian envelope tap wiring (N26 x N20)", () => { + it("taps one request and one response per exchange, paired by rpc_id", async () => { + await withGuardian(logIn, async (url, logPath) => { + await postRaw(url, JSON.stringify(toolCallEnvelope("rm -rf /", { id: 11 }))); + + const entries = readEntries(logPath); + expect(entries.length).toBe(2); + expect(entries[0]?.direction).toBe("request"); + expect(entries[1]?.direction).toBe("response"); + expect(entries[0]?.rpc_id).toBe(11); + expect(entries[1]?.rpc_id).toBe(11); + expect(entries[0]?.method).toBe("steps/toolCallRequest"); + expect(entries[1]?.method).toBe("steps/toolCallRequest"); + expect((entries[1]?.envelope as { result?: { decision?: string } }).result?.decision).toBe("deny"); + }); + }); + + it("taps handshake/hello in both directions", async () => { + await withGuardian(logIn, async (url, logPath) => { + await postRaw(url, JSON.stringify(makeEnvelope("handshake/hello", {}, { id: 42 }))); + + const entries = readEntries(logPath); + expect(entries.map((e) => e.direction)).toEqual(["request", "response"]); + expect(entries.every((e) => e.method === "handshake/hello")).toBe(true); + }); + }); + + // Decision P5. The envelope that fails validation is the most useful + // thing an ACS-first reader can see; tapping after the validator is + // exactly what would hide it. + it("taps a schema-invalid request, then its JSON-RPC error response", async () => { + await withGuardian(logIn, async (url, logPath) => { + const bad = toolCallEnvelope("rm -rf /", { id: 12 }); + delete (bad.params as Record).acs_version; + + await postRaw(url, JSON.stringify(bad)); + + const entries = readEntries(logPath); + expect(entries.length).toBe(2); + expect(entries[0]?.direction).toBe("request"); + expect((entries[0]?.envelope as { params: Record }).params.acs_version).toBeUndefined(); + const error = (entries[1]?.envelope as { error?: { code: number } }).error; + expect(error?.code).toBeLessThanOrEqual(-32000); + expect(error?.code).toBeGreaterThanOrEqual(-32099); + }); + }); + + it("taps an unparseable body as a lone response with rpc_id null -- no request line to pair with", async () => { + await withGuardian(logIn, async (url, logPath) => { + await postRaw(url, "{not json"); + + const entries = readEntries(logPath); + expect(entries.length).toBe(1); + expect(entries[0]?.direction).toBe("response"); + expect(entries[0]?.rpc_id).toBeNull(); + expect(entries[0]?.method).toBeNull(); + expect((entries[0]?.envelope as { error?: { code: number } }).error?.code).toBe(-32700); + }); + }); + + // Global constraint 8, end to end: the tap is on the decision path, so + // this is the test that says a broken tap cannot become a fail-open. + it("still denies rm -rf / when every tap write fails", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-tap-broken-")); + const blocker = join(dir, "blocker"); + writeFileSync(blocker, ""); + const guardian = await startGuardian({ + port: 0, + manifestPath: "policy/manifest.yaml", + envelopeLogPath: join(blocker, "nested", "envelopes.jsonl"), + }); + try { + const response = (await postRaw(guardian.url, JSON.stringify(toolCallEnvelope("rm -rf /")))) as { + result?: { decision?: string }; + error?: unknown; + }; + expect(response.error).toBeUndefined(); + expect(response.result?.decision).toBe("deny"); + } finally { + await guardian.close(); + unlinkSync(blocker); + rmdirSync(dir); + } + }); + + // Decision P3. + it("writes nothing when envelopeLogPath is omitted", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-tap-off-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml" }); + try { + await postRaw(guardian.url, JSON.stringify(toolCallEnvelope("ls -la"))); + expect(existsSync(logPath)).toBe(false); + } finally { + await guardian.close(); + rmdirSync(dir); + } + }); +}); +``` + +- [ ] **Step 2: Run it, expect FAIL** + +```bash +bun test packages/guardian/test/envelope-tap-wiring.test.ts +``` + +Expected: `envelopeLogPath` is not a known option, so nothing is written and the first assertion fails on `entries.length` being 0. + +- [ ] **Step 3: Minimal implementation** + +`packages/guardian/src/server.ts` — add the import, extend the options, create the tap, restructure `handleAcsRequest`: + +```ts +import { createEnvelopeTap, NULL_TAP, type EnvelopeTap } from "./envelope-tap.ts"; +``` + +```ts +export type StartGuardianOptions = { + port: number; + manifestPath: string; + mappingPath?: string; + /** Path to S6, the JSONL envelope log (N26). Omitted means no tap: every + * V1 test constructs Guardians freely and a default-on tap would scatter + * files through the working tree. `packages/guardian/src/main.ts` -- the + * demo path -- passes it. See the plan's decision P3. */ + envelopeLogPath?: string; +}; +``` + +```ts +export async function startGuardian({ + port, + manifestPath, + mappingPath, + envelopeLogPath, +}: StartGuardianOptions): Promise { + const bridge = createBridge(manifestPath); + const mapping = loadMapping(mappingPath ?? MAPPING_PATH); + const tap = envelopeLogPath ? createEnvelopeTap({ path: envelopeLogPath }) : NULL_TAP; + + const server = Bun.serve({ + port, + async fetch(req) { + const { pathname } = new URL(req.url); + if (req.method !== "POST" || pathname !== ACS_PATH) { + return new Response("Not Found", { status: 404 }); + } + const response = await handleAcsRequest(req, bridge, mapping, tap); + return Response.json(response); + }, + }); + + return { + url: `http://localhost:${server.port}${ACS_PATH}`, + async close() { + await server.stop(true); + }, + }; +} +``` + +Replace `handleAcsRequest` with the three-phase version, and move V1's body into `dispatch` unchanged: + +> **Corrected after V2's whole-branch review (findings 1 and 4).** The comment below miscounted, and the code inherited the miscount verbatim: `dispatch` leaves by **six** routes, not four — five `return`s plus the rethrow of any non-`EnvelopeValidationError` — so V3's N27 adds a seventh. Worse, the rethrow was the route the "structure guarantees totality" argument did not cover: nothing caught it, so `Bun.serve` answered with a `text/html` 500 that S6 never recorded and `res.json()` could not parse. Shipped code wraps the `dispatch` call in a try/catch that turns any escape into a well-formed JSON-RPC error (still not a decision — N27 stays V3), so every route out of `dispatch` now produces a response object and every response object gets tapped. + +```ts +/** + * Three phases, in order: parse, tap the request, dispatch, tap the + * response. The tap calls live here and only here -- `dispatch` below has + * four return sites and V3 adds a fifth (N27), so tapping inside it would + * make totality something a future task has to remember rather than + * something the structure guarantees. + * + * The tap itself is total (see envelope-tap.ts): these two calls cannot + * throw, so they cannot turn a governed tool call into an ungoverned one. + */ +async function handleAcsRequest( + req: Request, + bridge: ReturnType, + mapping: Mapping, + tap: EnvelopeTap, +): Promise { + let raw: unknown; + try { + raw = await req.json(); + } catch { + // Nothing parseable arrived, so there is no request envelope to tap -- + // the response is deliberately recorded unpaired, which is what the + // Inspector renders when a host sends a malformed body. + const parseError = errorResponse(null, -32700, "Parse error"); + tap.write("response", parseError, null); + return parseError; + } + + // Decision P5: before validation, so an envelope that fails the schema is + // visible to the Inspector rather than invisible. + const method = extractMethod(raw); + tap.write("request", raw, method); + + const response = await dispatch(raw, bridge, mapping); + tap.write("response", response, method); + return response; +} + +async function dispatch( + raw: unknown, + bridge: ReturnType, + mapping: Mapping, +): Promise { + const rpcId = extractId(raw); + + let envelope: ToolCallRequestEnvelope; + try { + envelope = validateEnvelope(raw); + } catch (error) { + if (error instanceof EnvelopeValidationError) { + return errorResponse(rpcId, ENVELOPE_INVALID_CODE, error.message, { pointer: error.pointer }); + } + throw error; + } + + if (envelope.method === HANDSHAKE_METHOD) { + return successResponse(envelope.id, handshakeResponder()); + } + + if (envelope.method === TOOL_CALL_REQUEST_METHOD) { + try { + const snapshot = assembleSnapshot(envelope); + const { verdict } = await bridge.evaluate("pre_tool_call", snapshot); + const decision = mapVerdict(verdict, mapping); + + const result: Record = { + type: "final", + acs_version: envelope.params.acs_version, + request_id: envelope.params.request_id, + ...decision, + }; + return successResponse(envelope.id, result); + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + return errorResponse(rpcId, EVALUATION_FAILED_CODE, `evaluation failed: ${message}`); + } + } + + return errorResponse(rpcId, METHOD_NOT_DISPATCHED_CODE, `method not dispatched by this Guardian: ${envelope.method}`, { + method: envelope.method, + }); +} +``` + +Add beside `extractId`: + +```ts +/** Best-effort method name for tap labelling only. Never used to dispatch -- + * `dispatch` reads the schema-validated envelope's own `method`. */ +function extractMethod(raw: unknown): string | null { + if (typeof raw === "object" && raw !== null && "method" in raw) { + const method = (raw as { method: unknown }).method; + if (typeof method === "string") { + return method; + } + } + return null; +} +``` + +`packages/guardian/src/index.ts` — append: + +```ts +export { + createEnvelopeTap, + extractRpcId, + NULL_TAP, + type CreateEnvelopeTapOptions, + type EnvelopeTap, + type TapDirection, + type TapEntry, +} from "./envelope-tap.ts"; +``` + +`packages/guardian/src/main.ts` — add the constant, the env read, and pass it. Also print the log path, so the operator knows what to point the Inspector at: + +```ts +const DEFAULT_ENVELOPE_LOG = ".acs/envelopes.jsonl"; +``` + +```ts +const envelopeLogPath = process.env.ACS_ENVELOPE_LOG ?? DEFAULT_ENVELOPE_LOG; + +const guardian = await startGuardian({ port, manifestPath, envelopeLogPath }); +console.log(`Guardian listening at ${guardian.url}`); +console.log(`Envelope log (S6): ${envelopeLogPath}`); +``` + +`.gitignore` — append: + +``` +# S6, the ACS envelope log (N26). Local demo artifact; carries raw tool +# arguments verbatim, so it is never committed. +.acs/ +``` + +- [ ] **Step 4: Run it, expect PASS** + +```bash +bun test packages/guardian +bun run typecheck +``` + +V1's `packages/guardian/test/server.test.ts` must still pass unmodified — this task moved code and added a tap, it changed no decision and no error code. + +- [ ] **Step 5: Commit** + +``` +Tap every ACS envelope crossing the Guardian's wire (N26, N20, S6) + +Slice: #3 +Affordances: N26, N20, S6 +``` + +--- + +### Task 3: `tailEnvelopeLog()` and the Inspector package · slice #3 · N50 + +**Files:** +- Create: `packages/inspector/package.json` +- Create: `packages/inspector/src/tail-envelope-log.ts` +- Test: `packages/inspector/test/tail-envelope-log.test.ts` + +**Interfaces:** +- Produces: `tailEnvelopeLog(options): AsyncGenerator` and the Inspector's **own** `TapEntry` / `TapDirection` types. Task 4's renderers consume `TapEntry`; Task 5's CLI consumes `tailEnvelopeLog`. +- **Does not consume anything from `guardian` or `agt-bridge`** (global constraint 10). `TapEntry` is re-declared here on purpose — Task 5's round-trip test is what keeps the two declarations in agreement. + +- [ ] **Step 1: Write the failing test** — `packages/inspector/test/tail-envelope-log.test.ts` + +```ts +import { describe, expect, it } from "bun:test"; +import { appendFileSync, mkdtempSync, rmdirSync, truncateSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { tailEnvelopeLog, type TapEntry } from "../src/tail-envelope-log.ts"; + +const POLL_MS = 10; + +function entryLine(seq: number, direction: "request" | "response"): string { + const entry: TapEntry = { + seq, + recorded_at: "2026-08-09T12:04:31.221Z", + direction, + method: "steps/toolCallRequest", + rpc_id: seq, + envelope: { jsonrpc: "2.0", id: seq }, + }; + return `${JSON.stringify(entry)}\n`; +} + +/** Collects `count` entries or rejects after `timeoutMs`, then aborts the + * generator so the test cannot hang the suite. */ +async function collect( + iterable: AsyncGenerator, + count: number, + controller: AbortController, + timeoutMs = 3000, +): Promise { + const out: TapEntry[] = []; + const deadline = setTimeout(() => controller.abort(), timeoutMs); + try { + for await (const entry of iterable) { + out.push(entry); + if (out.length >= count) { + break; + } + } + } finally { + clearTimeout(deadline); + controller.abort(); + } + return out; +} + +function withTempDir(run: (dir: string, path: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-tail-")); + const path = join(dir, "envelopes.jsonl"); + return run(dir, path).finally(() => { + try { + unlinkSync(path); + } catch { + // the not-yet-created case never writes one + } + rmdirSync(dir); + }); +} + +describe("tailEnvelopeLog (N50)", () => { + it("yields entries appended after the tail starts, skipping what was already there", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, pollMs: POLL_MS, signal: controller.signal }); + // Give the generator a poll to record its starting offset before the + // append lands, which is the behaviour under test. + await Bun.sleep(POLL_MS * 3); + appendFileSync(path, entryLine(2, "response")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([2]); + }); + }); + + it("yields pre-existing entries when fromStart is set", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request") + entryLine(2, "response")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + const entries = await collect(tail, 2, controller); + expect(entries.map((e) => e.seq)).toEqual([1, 2]); + expect(entries[0]?.direction).toBe("request"); + }); + }); + + it("waits for a log file that does not exist yet", async () => { + await withTempDir(async (_dir, path) => { + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + await Bun.sleep(POLL_MS * 3); + writeFileSync(path, entryLine(1, "request")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([1]); + }); + }); + + it("reassembles a line delivered in two chunks", async () => { + await withTempDir(async (_dir, path) => { + const line = entryLine(1, "request"); + const split = Math.floor(line.length / 2); + writeFileSync(path, ""); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + appendFileSync(path, line.slice(0, split)); + await Bun.sleep(POLL_MS * 3); + appendFileSync(path, line.slice(split)); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([1]); + }); + }); + + it("restarts from zero when the log is truncated underneath it", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + const first = await collectOne(tail); + expect(first?.seq).toBe(1); + + truncateSync(path, 0); + await Bun.sleep(POLL_MS * 3); + appendFileSync(path, entryLine(9, "response")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([9]); + }); + + async function collectOne(tail: AsyncGenerator): Promise { + const { value } = await tail.next(); + return value ?? undefined; + } + }); + + it("reports a malformed line and keeps streaming", async () => { + await withTempDir(async (_dir, path) => { + const malformed: string[] = []; + writeFileSync(path, "{not json\n" + entryLine(3, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([3]); + expect(malformed).toEqual(["{not json"]); + }); + }); + + it("ends when the signal aborts", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, ""); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + controller.abort(); + + const entries: TapEntry[] = []; + for await (const entry of tail) { + entries.push(entry); + } + expect(entries).toEqual([]); + }); + }); +}); +``` + +- [ ] **Step 2: Run it, expect FAIL** + +```bash +bun test packages/inspector +``` + +Expected: the module does not exist. + +- [ ] **Step 3: Minimal implementation** + +`packages/inspector/package.json`: + +```json +{ + "name": "inspector", + "private": true, + "type": "module", + "version": "0.1.0", + "main": "src/index.ts", + "types": "src/index.ts" +} +``` + +No `dependencies` block at all — global constraints 10 and 12. Run `bun install` once so the workspace links it. + +`packages/inspector/src/tail-envelope-log.ts`: + +```ts +/** + * tailEnvelopeLog (N50) streams S6 -- the Guardian's JSONL envelope log -- + * as it grows, the way `tail -f` does. + * + * This package deliberately imports nothing from `guardian` or from + * `agt-bridge` (global constraint 10). The Inspector reads a file that the + * Guardian happens to write; it holds no compile-time knowledge of the + * process that produced it, which is the point of R5.1 -- envelopes are + * inspectable *on the wire*, not through our own type graph. TapEntry is + * therefore re-declared here rather than imported. The round-trip test at + * test/envelope-tap-roundtrip.test.ts is what keeps the two declarations in + * agreement; if they drift, it fails. + * + * Polling rather than fs.watch: appends to a growing file are exactly the + * case where watch semantics differ most across platforms, and a 120ms poll + * on a local demo log costs nothing. + */ +import { closeSync, existsSync, openSync, readSync, statSync } from "node:fs"; + +export type TapDirection = "request" | "response"; + +/** One line of S6, as written by the Guardian's envelope tap. */ +export type TapEntry = { + seq: number; + recorded_at: string; + direction: TapDirection; + method: string | null; + rpc_id: string | number | null; + envelope: unknown; +}; + +export type TailOptions = { + path: string; + /** Replay everything already in the file before following. Default false: + * start at the current end, like `tail -f`. */ + fromStart?: boolean; + pollMs?: number; + signal?: AbortSignal; + /** Called per unparseable line. Defaults to one stderr warning. Streaming + * continues either way -- a corrupt line is not a reason to stop showing + * the ones after it. */ + onMalformedLine?: (line: string, error: unknown) => void; +}; + +const NEWLINE = 0x0a; + +export async function* tailEnvelopeLog({ + path, + fromStart = false, + pollMs = 120, + signal, + onMalformedLine = warnMalformedLine, +}: TailOptions): AsyncGenerator { + let offset = fromStart ? 0 : sizeOf(path); + // Bytes, not a string: a poll can land mid-line and, worse, mid-codepoint. + // Decoding only complete lines keeps multi-byte UTF-8 intact. + let pending = Buffer.alloc(0); + + while (!signal?.aborted) { + const size = sizeOf(path); + + if (size < offset) { + // Truncated or rotated underneath us (`: > .acs/envelopes.jsonl`). + offset = 0; + pending = Buffer.alloc(0); + } + + if (size > offset) { + pending = Buffer.concat([pending, readRange(path, offset, size - offset)]); + offset = size; + + let newline = pending.indexOf(NEWLINE); + while (newline !== -1) { + const line = pending.subarray(0, newline).toString("utf8"); + pending = pending.subarray(newline + 1); + newline = pending.indexOf(NEWLINE); + + if (line.trim() === "") { + continue; + } + let entry: TapEntry; + try { + entry = JSON.parse(line) as TapEntry; + } catch (error) { + onMalformedLine(line, error); + continue; + } + yield entry; + } + } + + if (signal?.aborted) { + return; + } + await sleep(pollMs, signal); + } +} + +function sizeOf(path: string): number { + if (!existsSync(path)) { + return 0; + } + try { + return statSync(path).size; + } catch { + return 0; + } +} + +function readRange(path: string, offset: number, length: number): Buffer { + const buffer = Buffer.alloc(length); + const fd = openSync(path, "r"); + try { + const read = readSync(fd, buffer, 0, length, offset); + return buffer.subarray(0, read); + } finally { + closeSync(fd); + } +} + +function warnMalformedLine(line: string, error: unknown): void { + const message = error instanceof Error ? error.message : String(error); + console.error(`skipping unparseable envelope-log line (${message}): ${line.slice(0, 120)}`); +} + +function sleep(ms: number, signal?: AbortSignal): Promise { + return new Promise((resolve) => { + const timer = setTimeout(() => { + signal?.removeEventListener("abort", onAbort); + resolve(); + }, ms); + function onAbort(): void { + clearTimeout(timer); + resolve(); + } + signal?.addEventListener("abort", onAbort, { once: true }); + }); +} +``` + +- [ ] **Step 4: Run it, expect PASS** + +```bash +bun install +bun test packages/inspector +bun run typecheck +``` + +- [ ] **Step 5: Commit** + +``` +Add the Inspector package and tailEnvelopeLog (N50) + +Slice: #3 +Affordances: N50 +``` + +--- + +### Task 4: The envelope stream and the decision badge · slice #3 · U20, U21 + +**Files:** +- Create: `packages/inspector/src/render.ts` +- Test: `packages/inspector/test/render.test.ts` + +**Interfaces:** +- Consumes: `TapEntry` from `./tail-envelope-log.ts` (Task 3). +- Produces: `renderEntry(entry, options?): string` (U20) and `renderDecisionBadge(entry, options?): string | null` (U21), plus `type RenderOptions = { color?: boolean; indent?: number }`. Task 5's CLI calls `renderEntry`. + +**Requirements:** + +Both functions are pure — no `process`, no env reads, no clock. `color` defaults to `false` so tests assert plain strings; the CLI decides whether the terminal wants ANSI. + +`renderDecisionBadge` returns `null` for requests and for responses that carry no decision (a ServerHello is the live example). Its states: + +| Response shape | Badge | Colour | +|---|---|---| +| `error` present | `✖ ERROR ` + message | red | +| `result.decision === "deny"` | `● DENY` | red | +| `result.decision === "allow"` with non-empty `policy_references` | `◐ ALLOW (policy fired — ACS "warn")` | yellow | +| `result.decision === "allow"` | `○ ALLOW` | green | +| any other decision (`modify`, `ask`, `defer`) | `◆ ` | cyan | +| neither `result.decision` nor `error` | `null` | — | + +The fourth row against the third is the slice's own reason for U21: §V2 says a `warn` arrives as `allow` with a non-empty `policy_references`, "and the badge is what makes that legible rather than buried". A badge that renders both as `ALLOW` fails this slice. + +`reason_codes` and `policy_references` are appended when present, as `reason_codes=[a, b]` and `policy_references=[policy_id#rule_id]`. + +- [ ] **Step 1: Write the failing test** — `packages/inspector/test/render.test.ts` + +```ts +import { describe, expect, it } from "bun:test"; +import { renderDecisionBadge, renderEntry } from "../src/render.ts"; +import type { TapEntry } from "../src/tail-envelope-log.ts"; + +function entry(overrides: Partial): TapEntry { + return { + seq: 3, + recorded_at: "2026-08-09T12:04:31.221Z", + direction: "response", + method: "steps/toolCallRequest", + rpc_id: 1, + envelope: {}, + ...overrides, + }; +} + +function response(result: Record): TapEntry { + return entry({ envelope: { jsonrpc: "2.0", id: 1, result } }); +} + +describe("renderDecisionBadge (U21)", () => { + it("returns null for requests", () => { + expect(renderDecisionBadge(entry({ direction: "request", envelope: { jsonrpc: "2.0", id: 1 } }))).toBeNull(); + }); + + it("returns null for a response with no decision -- a ServerHello", () => { + expect(renderDecisionBadge(response({ negotiated_version: "0.1.0", on_decision_failure: "proceed" }))).toBeNull(); + }); + + it("badges a deny, with reason_codes and policy_references", () => { + const badge = renderDecisionBadge( + response({ + decision: "deny", + reason_codes: ["destructive_shell_command_blocked"], + policy_references: [{ policy_id: "agt_stock", rule_id: "destructive_shell_command_blocked" }], + }), + ); + + expect(badge).toBe( + "● DENY reason_codes=[destructive_shell_command_blocked] " + + "policy_references=[agt_stock#destructive_shell_command_blocked]", + ); + }); + + it("badges a plain allow", () => { + expect(renderDecisionBadge(response({ decision: "allow" }))).toBe("○ ALLOW"); + }); + + // The reason U21 exists, per the slices doc: an AGT `warn` arrives as an + // ACS `allow` with a non-empty policy_references, and the badge is what + // keeps it from being buried. + it("distinguishes an allow that carries policy_references -- ACS's encoding of warn", () => { + const badge = renderDecisionBadge( + response({ + decision: "allow", + reason_codes: ["drift_detected"], + policy_references: [{ policy_id: "agt_stock", rule_id: "drift_detected" }], + }), + ); + + expect(badge).toBe( + '◐ ALLOW (policy fired — ACS "warn") reason_codes=[drift_detected] ' + + "policy_references=[agt_stock#drift_detected]", + ); + expect(badge).not.toBe(renderDecisionBadge(response({ decision: "allow" }))); + }); + + it("badges modify, ask, and defer", () => { + expect(renderDecisionBadge(response({ decision: "modify" }))).toBe("◆ MODIFY"); + expect(renderDecisionBadge(response({ decision: "ask" }))).toBe("◆ ASK"); + expect(renderDecisionBadge(response({ decision: "defer" }))).toBe("◆ DEFER"); + }); + + it("badges a JSON-RPC error", () => { + const badge = renderDecisionBadge( + entry({ envelope: { jsonrpc: "2.0", id: 1, error: { code: -32010, message: "ACS envelope failed" } } }), + ); + + expect(badge).toBe("✖ ERROR -32010 ACS envelope failed"); + }); + + it("emits ANSI only when colour is asked for", () => { + const plain = renderDecisionBadge(response({ decision: "deny" }), { color: false }); + const coloured = renderDecisionBadge(response({ decision: "deny" }), { color: true }); + + expect(plain).toBe("● DENY"); + expect(coloured).toContain("\u001b["); + expect(coloured).toContain("DENY"); + }); +}); + +describe("renderEntry (U20)", () => { + it("renders a request as a header line plus pretty JSON, with no badge", () => { + const rendered = renderEntry( + entry({ + direction: "request", + seq: 1, + envelope: { jsonrpc: "2.0", method: "steps/toolCallRequest", id: 1 }, + }), + ); + + expect(rendered.split("\n")[0]).toBe("── #1 12:04:31.221 → REQUEST steps/toolCallRequest id=1"); + expect(rendered).toContain('"jsonrpc": "2.0"'); + expect(rendered).not.toContain("●"); + }); + + it("renders a response as a header line, a badge line, then pretty JSON", () => { + const rendered = renderEntry(response({ decision: "deny", reason_codes: ["blocked"] })); + const lines = rendered.split("\n"); + + expect(lines[0]).toBe("── #3 12:04:31.221 ← RESPONSE steps/toolCallRequest id=1"); + expect(lines[1]).toBe("● DENY reason_codes=[blocked]"); + expect(lines[2]).toBe("{"); + }); + + it("labels an unpaired response -- the malformed-body case -- without an id or a method", () => { + const rendered = renderEntry( + entry({ method: null, rpc_id: null, envelope: { jsonrpc: "2.0", id: null, error: { code: -32700, message: "Parse error" } } }), + ); + + expect(rendered.split("\n")[0]).toBe("── #3 12:04:31.221 ← RESPONSE (no method) (unpaired)"); + }); + + it("keeps the envelope verbatim -- pretty-printing only reshapes whitespace", () => { + const envelope = { jsonrpc: "2.0", id: 1, result: { decision: "allow", nested: { deep: [1, 2] } } }; + const rendered = renderEntry(entry({ envelope })); + const jsonStart = rendered.indexOf("{"); + + expect(JSON.parse(rendered.slice(jsonStart))).toEqual(envelope); + }); +}); +``` + +- [ ] **Step 2: Run it, expect FAIL** + +```bash +bun test packages/inspector/test/render.test.ts +``` + +- [ ] **Step 3: Minimal implementation** — `packages/inspector/src/render.ts` + +```ts +/** + * U20 (envelope stream) and U21 (decision badge). + * + * Both functions are pure: no clock, no env, no process. The CLI decides + * whether the terminal wants ANSI and passes `color`; tests assert exact + * plain strings. Nothing here knows what produced a decision -- the badge + * reads ACS's own `decision`, `reason_codes`, and `policy_references` + * fields and nothing else (global constraint 9). + */ +import type { TapEntry } from "./tail-envelope-log.ts"; + +export type RenderOptions = { color?: boolean; indent?: number }; + +const RESET = "\u001b[0m"; +const RED = "\u001b[31m"; +const GREEN = "\u001b[32m"; +const YELLOW = "\u001b[33m"; +const CYAN = "\u001b[36m"; +const DIM = "\u001b[2m"; + +type PolicyReference = { policy_id?: string; policy_version?: string; rule_id?: string }; +type DecisionResult = { + decision?: unknown; + reason_codes?: unknown; + policy_references?: unknown; +}; +type ResponseEnvelope = { result?: DecisionResult; error?: { code?: unknown; message?: unknown } }; + +function paint(text: string, color: string, enabled: boolean): string { + return enabled ? `${color}${text}${RESET}` : text; +} + +/** `2026-08-09T12:04:31.221Z` -> `12:04:31.221`. Sliced, not parsed: UTC and + * locale-independent, so rendered output is the same everywhere. */ +function clockOf(recordedAt: string): string { + const time = recordedAt.slice(11, 23); + return time.length === 12 ? time : recordedAt; +} + +function stringList(value: unknown): string[] { + return Array.isArray(value) ? value.filter((item): item is string => typeof item === "string") : []; +} + +function referenceList(value: unknown): string[] { + if (!Array.isArray(value)) { + return []; + } + return value + .filter((item): item is PolicyReference => typeof item === "object" && item !== null) + .map((ref) => (ref.rule_id ? `${ref.policy_id ?? "?"}#${ref.rule_id}` : `${ref.policy_id ?? "?"}`)); +} + +/** U21. Null when this entry carries no decision and no error: a request, or + * a response such as a ServerHello. */ +export function renderDecisionBadge(entry: TapEntry, options: RenderOptions = {}): string | null { + if (entry.direction !== "response") { + return null; + } + const color = options.color ?? false; + const envelope = (typeof entry.envelope === "object" && entry.envelope !== null ? entry.envelope : {}) as ResponseEnvelope; + + if (envelope.error) { + const code = typeof envelope.error.code === "number" ? envelope.error.code : "?"; + const message = typeof envelope.error.message === "string" ? envelope.error.message : ""; + return paint(`✖ ERROR ${code}${message ? ` ${message}` : ""}`, RED, color); + } + + const result = envelope.result; + if (!result || typeof result.decision !== "string") { + return null; + } + + const reasonCodes = stringList(result.reason_codes); + const references = referenceList(result.policy_references); + + let head: string; + if (result.decision === "deny") { + head = paint("● DENY", RED, color); + } else if (result.decision === "allow" && references.length > 0) { + // ACS has no `warn`; a policy that fired but let the action proceed + // arrives as `allow` with a non-empty policy_references. Rendering it + // identically to a clean allow is exactly what this badge exists to + // prevent (slices doc, §V2). + head = paint('◐ ALLOW (policy fired — ACS "warn")', YELLOW, color); + } else if (result.decision === "allow") { + head = paint("○ ALLOW", GREEN, color); + } else { + head = paint(`◆ ${result.decision.toUpperCase()}`, CYAN, color); + } + + const parts = [head]; + if (reasonCodes.length > 0) { + parts.push(`reason_codes=[${reasonCodes.join(", ")}]`); + } + if (references.length > 0) { + parts.push(`policy_references=[${references.join(", ")}]`); + } + return parts.join(" "); +} + +/** U20. Header line, optional badge line, then the envelope as pretty JSON -- + * the same bytes that crossed the wire, only re-indented. */ +export function renderEntry(entry: TapEntry, options: RenderOptions = {}): string { + const color = options.color ?? false; + const arrow = entry.direction === "request" ? "→ REQUEST " : "← RESPONSE"; + const method = entry.method ?? "(no method)"; + const id = entry.rpc_id === null ? "(unpaired)" : `id=${entry.rpc_id}`; + + const header = paint(`── #${entry.seq} ${clockOf(entry.recorded_at)} ${arrow} ${method} ${id}`, DIM, color); + const badge = renderDecisionBadge(entry, options); + const body = JSON.stringify(entry.envelope, null, options.indent ?? 2); + + return [header, ...(badge === null ? [] : [badge]), body].join("\n"); +} +``` + +- [ ] **Step 4: Run it, expect PASS** + +```bash +bun test packages/inspector +bun run typecheck +``` + +- [ ] **Step 5: Commit** + +``` +Render the envelope stream and the decision badge (U20, U21) + +Slice: #3 +Affordances: U20, U21 +``` + +--- + +### Task 5: The Inspector CLI and the tap↔tail contract test · slice #3 · U20, U21, N50, N26 + +**Files:** +- Create: `packages/inspector/src/main.ts` +- Create: `packages/inspector/src/index.ts` +- Modify: `package.json` (add the `inspector` script) +- Test: `test/envelope-tap-roundtrip.test.ts` + +**Interfaces:** +- Consumes: `tailEnvelopeLog` (Task 3), `renderEntry` (Task 4), `startGuardian` with `envelopeLogPath` (Task 2). +- Produces: `bun run inspector`. + +**Requirements:** + +The round-trip test is the point of this task, not the CLI. Constraint 10 has the Inspector declaring its own `TapEntry` instead of importing the Guardian's; that duplication is only safe while something fails when they drift. This test writes through the real Guardian tap and reads through the real Inspector tail, so a field renamed on either side breaks it. + +- [ ] **Step 1: Write the failing test** — `test/envelope-tap-roundtrip.test.ts` + +```ts +import { describe, expect, it } from "bun:test"; +import { mkdtempSync, rmdirSync, unlinkSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startGuardian } from "../packages/guardian/src/index.ts"; +import { tailEnvelopeLog, type TapEntry } from "../packages/inspector/src/tail-envelope-log.ts"; +import { renderDecisionBadge } from "../packages/inspector/src/render.ts"; + +/** + * The contract test for S6. The Guardian writes the log; the Inspector + * declares its own TapEntry and reads it back (global constraint 10). If + * either side renames a field, adds a required one, or changes a type, this + * is what fails -- nothing else would, because the two never share a type. + */ +function toolCallEnvelope(command: string, id: number): Record { + return { + jsonrpc: "2.0", + method: "steps/toolCallRequest", + id, + params: { + acs_version: "0.1.0", + request_id: crypto.randomUUID(), + timestamp: new Date().toISOString(), + metadata: { agent_id: "agent-1", session_id: crypto.randomUUID() }, + payload: { tool: { name: "run_shell" }, arguments: { command: { value: command } } }, + }, + }; +} + +async function take( + tail: AsyncGenerator, + count: number, + controller: AbortController, +): Promise { + const out: TapEntry[] = []; + const deadline = setTimeout(() => controller.abort(), 5000); + try { + for await (const entry of tail) { + out.push(entry); + if (out.length >= count) { + break; + } + } + } finally { + clearTimeout(deadline); + controller.abort(); + } + return out; +} + +describe("S6 round trip: Guardian tap (N26) -> Inspector tail (N50) -> badge (U21)", () => { + it("a denied tool call arrives as a paired request/response the Inspector can render", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-roundtrip-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ + port: 0, + manifestPath: "policy/manifest.yaml", + envelopeLogPath: logPath, + }); + const controller = new AbortController(); + + try { + await fetch(guardian.url, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(toolCallEnvelope("rm -rf /", 77)), + }); + + const tail = tailEnvelopeLog({ path: logPath, fromStart: true, pollMs: 10, signal: controller.signal }); + const [request, response] = await take(tail, 2, controller); + + // Every field the Inspector's TapEntry declares must actually be + // present and correctly typed on what the Guardian wrote. + expect(request?.seq).toBe(1); + expect(response?.seq).toBe(2); + expect(typeof request?.recorded_at).toBe("string"); + expect(request?.direction).toBe("request"); + expect(response?.direction).toBe("response"); + expect(request?.method).toBe("steps/toolCallRequest"); + expect(request?.rpc_id).toBe(77); + expect(response?.rpc_id).toBe(77); + + // ...and the badge reads a real AGT-backed decision off it. + expect(renderDecisionBadge(response as TapEntry)).toContain("DENY"); + } finally { + controller.abort(); + await guardian.close(); + unlinkSync(logPath); + rmdirSync(dir); + } + }); +}); +``` + +- [ ] **Step 2: Run it, expect FAIL** + +```bash +bun test test/envelope-tap-roundtrip.test.ts +``` + +- [ ] **Step 3: Minimal implementation** + +`packages/inspector/src/index.ts`: + +```ts +/** Public surface of the inspector package. */ +export { tailEnvelopeLog, type TailOptions, type TapDirection, type TapEntry } from "./tail-envelope-log.ts"; +export { renderDecisionBadge, renderEntry, type RenderOptions } from "./render.ts"; +``` + +`packages/inspector/src/main.ts`: + +```ts +/** + * The Envelope Inspector's entrypoint -- `bun run inspector`. + * + * A third terminal beside `bun run guardian` and `claude`: it tails S6 and + * prints each ACS envelope as it crosses the wire. Not re-exported from + * ./index.ts -- this is a process entrypoint, not a library call. + * + * `ACS_ENVELOPE_LOG` defaults to `.acs/envelopes.jsonl`, the same default + * packages/guardian/src/main.ts writes to, so the two agree without either + * hardcoding the other's value. + */ +import { tailEnvelopeLog } from "./tail-envelope-log.ts"; +import { renderEntry } from "./render.ts"; + +const DEFAULT_ENVELOPE_LOG = ".acs/envelopes.jsonl"; + +const argv = process.argv.slice(2); +const fromStart = argv.includes("--from-start"); +const pathFlag = argv.indexOf("--path"); +const flagValue = pathFlag === -1 ? undefined : argv[pathFlag + 1]; + +if (pathFlag !== -1 && (flagValue === undefined || flagValue.startsWith("--"))) { + console.error("usage: bun run inspector -- [--from-start] [--path ]"); + process.exit(2); +} + +const path = flagValue ?? process.env.ACS_ENVELOPE_LOG ?? DEFAULT_ENVELOPE_LOG; +const color = Boolean(process.stdout.isTTY) && !process.env.NO_COLOR; + +const controller = new AbortController(); +process.on("SIGINT", () => controller.abort()); + +console.log(`Envelope Inspector — tailing ${path}${fromStart ? " (from the start)" : ""}`); +console.log("Ctrl-C to stop.\n"); + +for await (const entry of tailEnvelopeLog({ path, fromStart, signal: controller.signal })) { + console.log(renderEntry(entry, { color })); + console.log(""); +} +``` + +Root `package.json` — add to `scripts`: + +```json +"inspector": "bun run packages/inspector/src/main.ts" +``` + +- [ ] **Step 4: Run it, expect PASS** + +```bash +bun test +bun run typecheck +``` + +Then confirm the CLI itself runs, against the tree's own log path: + +```bash +ACS_ENVELOPE_LOG=/dev/null timeout 2 bun run inspector || true +``` + +Expected: it prints the two header lines and waits — no crash, no stack trace. + +- [ ] **Step 5: Commit** + +``` +Add the Inspector CLI and the S6 round-trip contract test (U20, U21, N50) + +Slice: #3 +Affordances: U20, U21, N50, N26 +``` + +--- + +### Task 6: The R5.2 gate, the runbook, and a live run · slice #3 · R5.1, R5.2, R7.1 + +**Files:** +- Modify: `test/invariants.test.ts` (append two `it` blocks inside the existing `describe`) +- Create: `docs/demos/v2-runbook.md` +- Modify: `slices/v2/README.md` +- Modify: `README.md` (Quickstart, What this proves, Status, Verify) + +**Interfaces:** +- Consumes: `readSourceFiles` and `assertNoVocabulary`, already module-scoped in `test/invariants.test.ts`. + +**Requirements:** + +R5.2 — "an ACS-first reader can trace one action end to end without reading AGT source" — becomes a property of the code: the tool that renders the trace names neither AGT nor any host, and imports neither. V1 established that this project turns its architectural claims into grep gates rather than prose, and the same reasoning applies here. + +Then **run the thing**, all three processes, and write the runbook from what actually appeared — not from what this plan predicts. If the output differs from what is written here, the output is right and the runbook says what happened. + +- [ ] **Step 1: Write the failing test** — append inside `describe("architectural invariants", ...)` in `test/invariants.test.ts` + +```ts + /** + * R5.2 -- "an ACS-first reader can trace one action end to end without + * reading AGT source". The Inspector is that reader's tool, so the claim + * is only real if the tool itself knows nothing about AGT and nothing + * about any particular host: it renders ACS envelopes as data. Both term + * lists from the two gates above apply to it at once. + */ + it("the Envelope Inspector's source contains zero AGT vocabulary and zero host vocabulary", () => { + assertNoVocabulary("packages/inspector/src", [ + "agt", + "AgentControl", + "rego", + "opa", + "intervention_point", + "verdict", + "claude", + "opencode", + "hookSpecificOutput", + "permissionDecision", + "stdin", + ]); + }); + + /** + * R5.1 -- envelopes are inspectable *on the wire*. If the Inspector + * imported the Guardian's types, "inspectable" would be a claim about our + * own type graph instead: any third-party reader of S6 has only the file. + * So does this one. + */ + it("the Envelope Inspector imports nothing from the Guardian or the AGT bridge", () => { + for (const { file, code } of readSourceFiles("packages/inspector/src")) { + for (const spec of ["guardian", "agt-bridge"]) { + const found = new RegExp(`from\\s+["'][^"']*${spec}[^"']*["']`).test(code); + expect({ file, spec, found }).toEqual({ file, spec, found: false }); + } + } + }); +``` + +- [ ] **Step 2: Run it, expect PASS immediately, then prove the gate bites** + +```bash +bun test test/invariants.test.ts +``` + +A gate that has never failed is not known to work. Temporarily add `import { NULL_TAP } from "guardian";` to `packages/inspector/src/render.ts`, re-run, and confirm **both** new tests fail. Then revert it and confirm they pass again. Record both outcomes in the task report. + +- [ ] **Step 3: Run the demo live, then write it down** + +Three terminals, from a clean tree: + +```bash +# 1 +bun run guardian +# 2 +bun run inspector +# 3 +echo '{"session_id":"demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"rm -rf /"}}' \ + | bun run hosts/claude-code/acs-hook.ts +echo '{"session_id":"demo","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"ls -la"}}' \ + | bun run hosts/claude-code/acs-hook.ts +``` + +Then the same through a real `claude` session, with `hosts/claude-code/settings.json` copied into `.claude/settings.json` as V1's quickstart describes. + +Write `docs/demos/v2-runbook.md` from the captured output. It must contain: +- the three-terminal setup, in order, and why the Guardian starts first (it creates `.acs/`); +- **real** pasted Inspector output for the deny and for the allow — the actual bytes, not a reconstruction; +- the note that `.acs/envelopes.jsonl` carries raw tool arguments verbatim and is gitignored for that reason; +- `: > .acs/envelopes.jsonl` as the way to clear the log mid-demo, and that the Inspector picks up cleanly afterwards because `tailEnvelopeLog` resets on truncation; +- `bun run inspector -- --from-start` to replay a session already recorded; +- the honest boundary: a schema-invalid envelope shows as a JSON-RPC **error**, not a `deny` decision. N27 — the affordance that turns Guardian-side failures into decisions — is V3. Say so, and say where. + +Update `slices/v2/README.md` to V1's shape: demo sentence, master-doc link, affordances, "What this slice delivers", and what is explicitly *not* in it (U22 session chain → V6; U23 posture badge and N51 → V3). + +Update `README.md`: +- **Quickstart**: a third step for `bun run inspector`, and the Guardian's new `Envelope log (S6)` line in its startup output. +- **What this proves → Delivered**: a row for R5.1 — every hook firing is inspectable as an ACS envelope, in both directions, including envelopes that fail validation — pointing at `test/envelope-tap-roundtrip.test.ts` and `packages/guardian/test/envelope-tap-wiring.test.ts`. +- **Verify**: the test count, re-read from a real `bun test` run. Do not carry V1's number forward and do not estimate it. +- **Status**: V2 implemented; V3–V8 shaped but not started. + +- [ ] **Step 4: Run it, expect PASS** + +```bash +bun test +bun run typecheck +``` + +- [ ] **Step 5: Commit** + +``` +Gate the Inspector on R5.1/R5.2, add the V2 runbook and quickstart + +Slice: #3 +Affordances: U20, U21, N26, N50, S6 +``` + +--- + +## Risks + +| # | Risk | Handling | +|---|---|---| +| 1 | A partially-written line is read mid-append and mis-parsed | `tailEnvelopeLog` buffers **bytes** and decodes only up to a newline, so a split line — or a split UTF-8 codepoint — is held until it completes. Task 3 tests it directly. | +| 2 | The tap becomes a fourth fail-open | Global constraint 8, `createEnvelopeTap`'s total-by-construction design, and Task 2's end-to-end test asserting `rm -rf /` is still denied when every tap write fails. | +| 3 | S6 grows without bound | Accepted for V2 and recorded in the slices doc. It is a gitignored local demo artifact; `: > .acs/envelopes.jsonl` truncates it safely mid-run because the tail resets on truncation. Rotation is not built. | +| 4 | The Inspector's duplicated `TapEntry` silently drifts from the Guardian's | Task 5's round-trip test exercises both real implementations against one file. | +| 5 | S6 contains sensitive tool arguments | Constraint 11 says record the wire verbatim — a redacting tap would make the Inspector lie. Handled by gitignoring `.acs/` and stating it in the runbook and the README, not by filtering. | diff --git a/package.json b/package.json index e20357a6..fd6acac4 100644 --- a/package.json +++ b/package.json @@ -7,7 +7,8 @@ "test": "bun test", "typecheck": "tsc -p tsconfig.json --noEmit", "verify:pin": "bash scripts/verify-pin.sh", - "guardian": "bun run packages/guardian/src/main.ts" + "guardian": "bun run packages/guardian/src/main.ts", + "inspector": "bun run packages/inspector/src/main.ts" }, "dependencies": { "agent-control-specification": "0.3.1-beta.0" diff --git a/packages/guardian/src/envelope-log-sink.ts b/packages/guardian/src/envelope-log-sink.ts new file mode 100644 index 00000000..e83d80f2 --- /dev/null +++ b/packages/guardian/src/envelope-log-sink.ts @@ -0,0 +1,138 @@ +/** + * createEnvelopeLogSink writes a JSONL record of every ACS envelope that + * crosses this Guardian's wire, in both directions. The Envelope Inspector + * reads this file and nothing else -- see packages/inspector, which + * deliberately imports nothing from here. + * + * Named for the artifact it produces and the role it plays, not for the + * mechanism: it writes a log, it does not "tap a wire". + * + * What "records the envelope" means here, precisely: the sink is handed the + * JSON *value* the Guardian parsed, and writes it unmodified -- no field + * stripping, no redaction, no reordering of anything we control. It is not a + * byte-for-byte copy of the request body. The parse happens upstream in + * server.ts (`await req.json()`) and has already collapsed duplicate keys, + * canonicalised number literals, and hoisted integer-like object keys -- and + * `arguments` keys are host-controlled, so `{"0": ...}` is a real shape, not + * a hypothetical. Recording raw bytes instead would make `envelope` a string + * rather than a JSON value, costing the Inspector its pretty-printing and + * the round-trip contract test; the accurate sentence is the better trade. + * + * Total by construction. Every write is wrapped: a failure disables the sink + * for the process lifetime, reports once, and is never propagated to the + * caller. The sink sits on the decision path: an observability feature that + * can turn a governed tool call into an ungoverned one is the one failure + * mode this module must never have. Observability degrades; governance does + * not. + */ +import { appendFileSync, mkdirSync } from "node:fs"; +import { dirname } from "node:path"; + +/** Which side of the exchange one envelope-log line recorded. */ +export type EnvelopeLogDirection = "request" | "response"; + +/** + * One line of the envelope log, named for what was recorded rather than for + * what recorded it. `envelope` is the JSON-RPC object as parsed, unmodified + * -- request or response -- and every other field is Guardian-side context + * the wire does not carry: a sequence number so a reader can detect gaps, a + * timestamp, the direction, the ACS method (JSON-RPC responses carry none, + * so the Guardian supplies the one it dispatched), and the JSON-RPC id that + * pairs the two directions. + */ +export type EnvelopeLogEntry = { + seq: number; + recorded_at: string; + direction: EnvelopeLogDirection; + method: string | null; + rpc_id: string | number | null; + envelope: unknown; +}; + +/** Where envelope-log lines go: the Guardian-side writer role. */ +export type EnvelopeLogSink = { + write(direction: EnvelopeLogDirection, envelope: unknown, method: string | null): void; + readonly path: string | null; +}; + +export type CreateEnvelopeLogSinkOptions = { + path: string; + /** Injectable clock, so tests can assert recorded_at exactly. */ + now?: () => Date; + /** Called at most once, on the first failure. Defaults to one stderr line. */ + onError?: (error: unknown) => void; +}; + +/** The sink a Guardian gets when no envelope log path was configured: a + * composition root disabling observability gets a value of the same + * `EnvelopeLogSink` type, not a special case. */ +export const NULL_ENVELOPE_LOG_SINK: EnvelopeLogSink = { + path: null, + write(): void {}, +}; + +/** The JSON-RPC id, when it is a scalar. Both request and response envelopes + * carry `id` at the top level, so one extractor serves both directions. */ +export function extractRpcId(envelope: unknown): string | number | null { + if (typeof envelope === "object" && envelope !== null && "id" in envelope) { + const id = envelope.id; + if (typeof id === "string" || typeof id === "number") { + return id; + } + } + return null; +} + +export function createEnvelopeLogSink({ + path, + now = () => new Date(), + onError, +}: CreateEnvelopeLogSinkOptions): EnvelopeLogSink { + let seq = 0; + let disabled = false; + + const fail = (error: unknown): void => { + disabled = true; + try { + if (onError) { + onError(error); + return; + } + const message = error instanceof Error ? error.message : String(error); + console.error(`envelope log sink disabled after failure (${path}): ${message}`); + } catch { + // Silently swallow any error from the callback or console.error + // to maintain the total-by-construction guarantee + } + }; + + try { + mkdirSync(dirname(path), { recursive: true }); + } catch (error) { + fail(error); + } + + return { + path, + write(direction, envelope, method): void { + if (disabled) { + return; + } + try { + const entry: EnvelopeLogEntry = { + seq: seq + 1, + recorded_at: now().toISOString(), + direction, + method, + rpc_id: extractRpcId(envelope), + envelope, + }; + const line = `${JSON.stringify(entry)}\n`; + appendFileSync(path, line); + seq += 1; + } catch (error) { + fail(error); + } + }, + }; +} diff --git a/packages/guardian/src/index.ts b/packages/guardian/src/index.ts index cd4d2de2..022a9ab1 100644 --- a/packages/guardian/src/index.ts +++ b/packages/guardian/src/index.ts @@ -1,4 +1,31 @@ -/** Public surface of the guardian package. */ +/** + * Public surface of the guardian package: the governance verbs, and nothing + * else. + * + * The envelope log's writer (`./envelope-log-sink.ts`) is deliberately not + * re-exported. Observability is an internal detail of running a Guardian, + * not part of the vocabulary a consumer of this package speaks, and a + * barrel that mixed `startGuardian` / `mapVerdict` with + * `createEnvelopeLogSink` / `NULL_ENVELOPE_LOG_SINK` / `extractRpcId` / the + * entry types would make the second look like the first. The one affordance + * a consumer actually needs is `envelopeLogPath` on `StartGuardianOptions`, + * which stays exactly where it was: you tell a Guardian where to write its + * envelope log, you do not assemble the sink yourself. + * + * `./server.ts` imports the sink directly, and the two test files that + * exercise it on its own + * (`packages/guardian/test/envelope-log-sink*.test.ts`) import + * `../src/envelope-log-sink.ts` directly too, so nothing outside this + * package needs it re-exported here. If an external writer of the envelope + * log ever becomes a real need, the answer is a narrow subpath export, not + * putting it back in this barrel. + * + * The Inspector is unaffected either way: it imports nothing from this + * package, reads the envelope log purely as a file, and re-declares + * `EnvelopeLogEntry` on its own side. `test/invariants.test.ts` enforces + * that boundary, and `test/envelope-log-sink-roundtrip.test.ts` keeps the + * two declarations honest with each other. + */ export { startGuardian, type StartGuardianOptions, type StartedGuardian } from "./server.ts"; export { buildServerHello, type ServerHello } from "./handshake.ts"; export { diff --git a/packages/guardian/src/main.ts b/packages/guardian/src/main.ts index f46e6b32..809ac15e 100644 --- a/packages/guardian/src/main.ts +++ b/packages/guardian/src/main.ts @@ -22,10 +22,13 @@ import { startGuardian } from "./server.ts"; const DEFAULT_PORT = 8787; const DEFAULT_MANIFEST_PATH = "policy/manifest.yaml"; +const DEFAULT_ENVELOPE_LOG = ".acs/envelopes.jsonl"; const port = Number(process.env.ACS_GUARDIAN_PORT ?? DEFAULT_PORT); const hostname = process.env.ACS_GUARDIAN_HOST; const manifestPath = process.env.ACS_MANIFEST_PATH ?? DEFAULT_MANIFEST_PATH; +const envelopeLogPath = process.env.ACS_ENVELOPE_LOG ?? DEFAULT_ENVELOPE_LOG; -const guardian = await startGuardian({ port, hostname, manifestPath }); +const guardian = await startGuardian({ port, hostname, manifestPath, envelopeLogPath }); console.log(`Guardian listening at ${guardian.url}`); +console.log(`Envelope log: ${envelopeLogPath}`); diff --git a/packages/guardian/src/server.ts b/packages/guardian/src/server.ts index 1a9fae27..94135839 100644 --- a/packages/guardian/src/server.ts +++ b/packages/guardian/src/server.ts @@ -8,18 +8,29 @@ * assemblePreToolCallSnapshot -> bridge.evaluate at the intervention point * resolveInterventionPoint picked -> mapVerdict -> response envelope. * - * The middle three run inside a try/catch, and that catch is load-bearing. - * Were a throw to escape this handler, Bun.serve would answer with its default - * error page, which is HTML rather than JSON-RPC. The host's client calls - * res.json() unconditionally, so an HTML body raises a SyntaxError there - * instead of surfacing a JSON-RPC error; the hook's catch-all then exits 1 - * with nothing on stdout, which Claude Code reads as "the hook never fired" - * and allows the tool call through ungoverned. That is a fail-open in a - * governance tool, so the catch must stay. + * Every throw on this path is caught, in two places, because nothing may + * escape the fetch handler. Bun.serve would answer an unhandled rejection with + * its default error page, which is HTML rather than JSON-RPC. The host's + * client calls res.json() unconditionally, so an HTML body raises a + * SyntaxError there instead of surfacing a JSON-RPC error; the hook's + * catch-all then exits 1 with nothing on stdout, which Claude Code reads as + * "the hook never fired" and allows the tool call through ungoverned. That is + * a fail-open in a governance tool. * - * What the catch guarantees is only that a well-formed JSON-RPC error reaches - * the client. It deliberately does not turn the failure into an ACS `deny`: - * which disposition an evaluation failure should carry is a separate question. + * The two catches are: + * - Inside `dispatch`, around assemblePreToolCallSnapshot, bridge.evaluate + * and mapVerdict: the evaluation itself. + * - Around the whole `dispatch` call in `handleAcsRequest`, as the outer + * net. `dispatch` rethrows anything that is not an + * EnvelopeValidationError, and that rethrow is live: validateEnvelope + * builds its Ajv registry lazily, on the first request rather than at + * boot, so a tree cloned without `--recurse-submodules` starts cleanly + * and then turns every request into an HTML 500. The outer net also + * writes the failure response through the same envelope-log call as any + * other response. + * + * Neither catch turns the failure into an ACS `deny` decision: which + * disposition a Guardian-side failure should carry is a separate question. * * The bridge and the mapping table are both built once, when startGuardian is * called, rather than per request -- AGT is meant to be constructed at boot @@ -48,6 +59,7 @@ import { type AcsRequestEnvelope, } from "./validate-envelope.ts"; import { buildServerHello, type ServerHello } from "./handshake.ts"; +import { createEnvelopeLogSink, NULL_ENVELOPE_LOG_SINK, type EnvelopeLogSink } from "./envelope-log-sink.ts"; /** * Every snapshot message this Guardian can send an intervention point. One @@ -72,6 +84,73 @@ const ACS_PATH = "/acs"; * so the name would make which interfaces are listening a property of the * machine's resolver rather than of this line. */ const LOOPBACK_ONLY = "127.0.0.1"; +// This tree's own absolute root -- `packages/guardian/src` is always three +// directories under it, the same relationship MAPPING_PATH above relies on. +// Used only to redact it out of error text before that text leaves the +// Guardian; see toRepoRelativeMessage below. +const REPO_ROOT = fileURLToPath(new URL("../../../", import.meta.url)).replace(/[/\\]+$/, ""); +// The lookahead requires the root to be followed by a path separator or the +// end of the string, before the optional `[/\\]?` consumes one such +// separator. Without it, a *sibling* directory whose name merely extends +// the root (`ACS_reference_implementation_old`) matched too: the literal +// text of REPO_ROOT is a prefix of that name, so it stripped, leaving a +// misleading `_old/packages/spec` behind -- not a disclosure of this tree's +// own location, since it's a different directory entirely, but a +// diagnostic that then reads as if it were one. The lookahead makes that +// prefix match fail outright, so a message naming the sibling is left +// alone, in full. +const REPO_ROOT_PATTERN = new RegExp(`${REPO_ROOT.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}(?=[/\\\\]|$)[/\\\\]?`, "g"); + +/** + * Both catches in this module -- the outer net in handleAcsRequest and the + * evaluation-failure catch in dispatch -- surface a real error to the ACS + * client and, through the envelope log, onto disk. A real error's message + * (an ENOENT out of a missing schema directory, say) carries this machine's + * absolute filesystem path, e.g. + * `/Users/you/.../ACS_reference_implementation/packages/spec/acs/...`. + * That is diagnostic in a way this demo's value depends on, so rather than + * replace it with something generic, this function removes only the part + * that discloses where this tree sits on disk, leaving the repo-relative + * remainder (`packages/spec/acs/...`) intact. + * + * Takes the caught value itself, as `unknown`, rather than a pre-extracted + * string, and calls `String()` on the whole thing. `instanceof Error` does + * not guarantee `.message` is a string -- nothing stops it from being + * reassigned to `undefined`, a number, or anything else after construction + * -- and a throw from this function on the outer-net path has nothing above + * `handleAcsRequest` to catch it. Staying total for any `unknown`, exactly + * what a catch clause can hand it, is what keeps that path from becoming an + * unrecorded HTML 500. + * + * Exported -- unlike this module's other internals (`dispatch`, `extractId`, + * `extractMethod`, `errorResponse`) -- so the redaction above and this + * function's totality for non-`Error`, non-string-message, and + * otherwise-shaped `unknown` values can be asserted directly, rather than + * only through a real Guardian and an HTTP round trip. The same behaviour is + * also covered end to end, separately -- see server.test.ts's + * `withFakeValidateEnvelopeGuardian` tests. + * + * The whole body is wrapped in its own try/catch, including the + * `instanceof` check -- belt-and-braces, not a reaction to a live bug. + * Nothing in this tree currently throws an `Error` whose `.message` is a + * throwing accessor, a value whose `toString`/`valueOf` throws, or a + * `Proxy` that throws on `get` or on `getPrototypeOf` (which would defeat + * `instanceof Error` itself, since it walks the prototype chain through + * `[[GetPrototypeOf]]`, before `String()` below ever runs). They are guarded + * anyway because "unreachable today" should not be load-bearing for the one + * function whose entire job is upholding this module's contract that + * nothing escapes the outer net. The fallback string names the failure mode + * rather than guessing at a partial message, since the point is that + * nothing about the original error could be read at all. + */ +export function toRepoRelativeMessage(error: unknown): string { + try { + const message = String(error instanceof Error ? error.message : error); + return message.replace(REPO_ROOT_PATTERN, ""); + } catch { + return ""; + } +} /** * ACS reserves -32000..-32099 for application errors (Specification §17), @@ -86,9 +165,10 @@ const LOOPBACK_ONLY = "127.0.0.1"; */ const ENVELOPE_INVALID_CODE = -32010; const METHOD_NOT_DISPATCHED_CODE = -32011; -/** A throw from assemblePreToolCallSnapshot, bridge.evaluate, or mapVerdict -- - * mapVerdict's own require_policy_references check, say, or any AGT runtime - * error. The module header explains why this must never become dead code. */ +/** Any throw the Guardian did not turn into a response itself: mapVerdict's + * own require_policy_references check, an AGT runtime error, or -- through + * handleAcsRequest's outer net -- a failure to even build the schema registry. + * The module header explains why none of this may become dead code. */ const EVALUATION_FAILED_CODE = -32020; type JsonRpcSuccess = { jsonrpc: "2.0"; id: string | number; result: AcsFinalResult | ServerHello }; @@ -113,6 +193,11 @@ export type StartGuardianOptions = { * handleAcsRequest against a real bridge, without touching the mapping * every other consumer reads. Not meant for production use. */ mappingPath?: string; + /** Path to the JSONL envelope log. Omitted means no sink: tests construct + * Guardians freely, and a default-on sink would scatter files through the + * working tree. `packages/guardian/src/main.ts` -- the demo path -- + * passes it. */ + envelopeLogPath?: string; }; export type StartedGuardian = { url: string; close(): Promise }; @@ -121,10 +206,12 @@ export async function startGuardian({ hostname, manifestPath, mappingPath, + envelopeLogPath, }: StartGuardianOptions): Promise { // Construct the bridge once at boot, not per request. const bridge = createBridge(manifestPath); const mapping = loadMapping(mappingPath ?? MAPPING_PATH); + const envelopeLog = envelopeLogPath ? createEnvelopeLogSink({ path: envelopeLogPath }) : NULL_ENVELOPE_LOG_SINK; const server = Bun.serve({ hostname: hostname ?? LOOPBACK_ONLY, @@ -134,7 +221,7 @@ export async function startGuardian({ if (req.method !== "POST" || pathname !== ACS_PATH) { return new Response("Not Found", { status: 404 }); } - const response = await handleAcsRequest(req, bridge, mapping); + const response = await handleAcsRequest(req, bridge, mapping, envelopeLog); return Response.json(response); }, }); @@ -147,6 +234,22 @@ export async function startGuardian({ }; } +/** + * Three phases, in order: parse, record the request, dispatch, record the + * response. The envelope-log writes live here and only here: `dispatch` + * below leaves by six routes (five `return`s and one rethrow), and future + * dispatch outcomes will add more, so writing to the envelope log inside it + * would make totality something a future change has to remember rather than + * something the structure guarantees. + * + * That guarantee only holds if every route out of `dispatch` is covered, + * including the one that throws: the try/catch below ensures every route + * out of `dispatch` produces a response object, and every response object + * reaches the envelope log. + * + * The sink itself is total (see envelope-log-sink.ts): these two calls cannot + * throw, so they cannot turn a governed tool call into an ungoverned one. + */ async function handleAcsRequest( req: Request, // The role, not `ReturnType`: this handler depends on @@ -154,14 +257,46 @@ async function handleAcsRequest( // factory happens to return. bridge: PolicyBridge, mapping: Mapping, + envelopeLog: EnvelopeLogSink, ): Promise { let raw: unknown; try { raw = await req.json(); } catch { - return errorResponse(null, -32700, "Parse error"); + // Nothing parseable arrived, so there is no request envelope to record + // -- the response is deliberately recorded unpaired, which is what the + // Inspector renders when a host sends a malformed body. + const parseError = errorResponse(null, -32700, "Parse error"); + envelopeLog.write("response", parseError, null); + return parseError; } + // Recorded before validation, so an envelope that fails the schema is + // visible to the Inspector rather than invisible. + const method = extractMethod(raw); + envelopeLog.write("request", raw, method); + + let response: JsonRpcSuccess | JsonRpcFailure; + try { + response = await dispatch(raw, bridge, mapping); + } catch (error) { + // The outer net. Deliberately a bare JSON-RPC error, not an ACS `deny` + // decision -- turning a validation failure into an explicit deny is a + // separate concern. What this buys is that the client can parse the + // answer at all, and that the envelope log holds a response line paired + // with the request line above it. + const message = toRepoRelativeMessage(error); + response = errorResponse(extractId(raw), EVALUATION_FAILED_CODE, `guardian failed to handle the request: ${message}`); + } + envelopeLog.write("response", response, method); + return response; +} + +async function dispatch( + raw: unknown, + bridge: ReturnType, + mapping: Mapping, +): Promise { const rpcId = extractId(raw); let envelope: AcsRequestEnvelope; @@ -207,7 +342,7 @@ async function handleAcsRequest( // See the module header. Deliberately a bare JSON-RPC error rather than // an ACS `deny` decision -- all this guarantees is that the client gets // a parseable envelope back instead of an HTML 500. - const message = error instanceof Error ? error.message : String(error); + const message = toRepoRelativeMessage(error); return errorResponse(rpcId, EVALUATION_FAILED_CODE, `evaluation failed: ${message}`); } } @@ -240,10 +375,22 @@ function errorResponse( * successful response always uses the schema-validated envelope's id. */ function extractId(raw: unknown): string | number | null { if (typeof raw === "object" && raw !== null && "id" in raw) { - const id = (raw as { id: unknown }).id; + const id = raw.id; if (typeof id === "string" || typeof id === "number") { return id; } } return null; } + +/** Best-effort method name for envelope-log labelling only. Never used to + * dispatch -- `dispatch` reads the schema-validated envelope's own `method`. */ +function extractMethod(raw: unknown): string | null { + if (typeof raw === "object" && raw !== null && "method" in raw) { + const method = raw.method; + if (typeof method === "string") { + return method; + } + } + return null; +} diff --git a/packages/guardian/test/envelope-log-sink-wiring.test.ts b/packages/guardian/test/envelope-log-sink-wiring.test.ts new file mode 100644 index 00000000..ed908627 --- /dev/null +++ b/packages/guardian/test/envelope-log-sink-wiring.test.ts @@ -0,0 +1,189 @@ +import { describe, expect, it, spyOn } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startGuardian } from "../src/index.ts"; +import type { EnvelopeLogEntry } from "../src/envelope-log-sink.ts"; + +function makeEnvelope( + method: string, + payload: Record, + overrides: { id?: number; requestId?: string } = {}, +): Record { + const { id = 1, requestId = crypto.randomUUID() } = overrides; + return { + jsonrpc: "2.0", + method, + id, + params: { + acs_version: "0.1.0", + request_id: requestId, + timestamp: new Date().toISOString(), + metadata: { agent_id: "agent-1", session_id: crypto.randomUUID() }, + payload, + }, + }; +} + +function toolCallEnvelope(command: string, overrides: { id?: number } = {}) { + return makeEnvelope( + "steps/toolCallRequest", + { tool: { name: "run_shell" }, arguments: { command: { value: command } } }, + overrides, + ); +} + +function readEntries(path: string): EnvelopeLogEntry[] { + if (!existsSync(path)) { + return []; + } + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as EnvelopeLogEntry); +} + +/** Non-recursive cleanup, as in envelope-log-sink.test.ts. */ +async function withGuardian( + logPathFor: (dir: string) => string, + run: (url: string, logPath: string) => Promise, +): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-envelope-log-wiring-")); + const logPath = logPathFor(dir); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath }); + try { + await run(guardian.url, logPath); + } finally { + await guardian.close(); + try { + unlinkSync(logPath); + } catch { + // some tests deliberately make the path unwritable + } + try { + rmdirSync(dir); + } catch { + // a blocker file may remain; the assertions already covered what matters + } + } +} + +async function postRaw(url: string, body: string): Promise { + const res = await fetch(url, { method: "POST", headers: { "content-type": "application/json" }, body }); + return await res.json(); +} + +const logIn = (dir: string) => join(dir, "envelopes.jsonl"); + +describe("Guardian envelope log wiring", () => { + it("records one request and one response per exchange, paired by rpc_id", async () => { + await withGuardian(logIn, async (url, logPath) => { + await postRaw(url, JSON.stringify(toolCallEnvelope("rm -rf /", { id: 11 }))); + + const entries = readEntries(logPath); + expect(entries.length).toBe(2); + expect(entries[0]?.direction).toBe("request"); + expect(entries[1]?.direction).toBe("response"); + expect(entries[0]?.rpc_id).toBe(11); + expect(entries[1]?.rpc_id).toBe(11); + expect(entries[0]?.method).toBe("steps/toolCallRequest"); + expect(entries[1]?.method).toBe("steps/toolCallRequest"); + expect((entries[1]?.envelope as { result?: { decision?: string } }).result?.decision).toBe("deny"); + }); + }); + + it("records handshake/hello in both directions", async () => { + await withGuardian(logIn, async (url, logPath) => { + await postRaw(url, JSON.stringify(makeEnvelope("handshake/hello", {}, { id: 42 }))); + + const entries = readEntries(logPath); + expect(entries.map((e) => e.direction)).toEqual(["request", "response"]); + expect(entries.every((e) => e.method === "handshake/hello")).toBe(true); + }); + }); + + // The envelope that fails validation is the most useful thing an + // ACS-first reader can see; recording it after the validator is exactly + // what would hide it. + it("records a schema-invalid request, then its JSON-RPC error response", async () => { + await withGuardian(logIn, async (url, logPath) => { + const bad = toolCallEnvelope("rm -rf /", { id: 12 }); + delete (bad.params as Record).acs_version; + + await postRaw(url, JSON.stringify(bad)); + + const entries = readEntries(logPath); + expect(entries.length).toBe(2); + expect(entries[0]?.direction).toBe("request"); + expect((entries[0]?.envelope as { params: Record }).params.acs_version).toBeUndefined(); + const error = (entries[1]?.envelope as { error?: { code: number } }).error; + expect(error?.code).toBeLessThanOrEqual(-32000); + expect(error?.code).toBeGreaterThanOrEqual(-32099); + }); + }); + + it("records an unparseable body as a lone response with rpc_id null -- no request line to pair with", async () => { + await withGuardian(logIn, async (url, logPath) => { + await postRaw(url, "{not json"); + + const entries = readEntries(logPath); + expect(entries.length).toBe(1); + expect(entries[0]?.direction).toBe("response"); + expect(entries[0]?.rpc_id).toBeNull(); + expect(entries[0]?.method).toBeNull(); + expect((entries[0]?.envelope as { error?: { code: number } }).error?.code).toBe(-32700); + }); + }); + + // End to end: the sink is on the decision path, so this is the test that + // says a broken sink cannot become a fail-open. + // + // No `onError` is passed here, so this exercises the sink's *default* + // reporter -- a single `console.error` line -- rather than the + // onError-captured path envelope-log-sink.test.ts's "reports once, then goes + // quiet" test covers. Spied and silenced so a deliberately-broken sink + // does not print real stderr into a clean `bun test` run, and asserted + // on so "reports once" is checked at the call site instead of merely + // claimed. + it("still denies rm -rf / when every envelope-log write fails", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-envelope-log-broken-")); + const blocker = join(dir, "blocker"); + writeFileSync(blocker, ""); + const errorSpy = spyOn(console, "error").mockImplementation(() => {}); + try { + const guardian = await startGuardian({ + port: 0, + manifestPath: "policy/manifest.yaml", + envelopeLogPath: join(blocker, "nested", "envelopes.jsonl"), + }); + try { + const response = (await postRaw(guardian.url, JSON.stringify(toolCallEnvelope("rm -rf /")))) as { + result?: { decision?: string }; + error?: unknown; + }; + expect(response.error).toBeUndefined(); + expect(response.result?.decision).toBe("deny"); + expect(errorSpy).toHaveBeenCalledTimes(1); + } finally { + await guardian.close(); + } + } finally { + errorSpy.mockRestore(); + unlinkSync(blocker); + rmdirSync(dir); + } + }); + + it("writes nothing when envelopeLogPath is omitted", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-envelope-log-off-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml" }); + try { + await postRaw(guardian.url, JSON.stringify(toolCallEnvelope("ls -la"))); + expect(existsSync(logPath)).toBe(false); + } finally { + await guardian.close(); + rmdirSync(dir); + } + }); +}); diff --git a/packages/guardian/test/envelope-log-sink.test.ts b/packages/guardian/test/envelope-log-sink.test.ts new file mode 100644 index 00000000..1424bd8c --- /dev/null +++ b/packages/guardian/test/envelope-log-sink.test.ts @@ -0,0 +1,186 @@ +import { describe, expect, it } from "bun:test"; +import { mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + createEnvelopeLogSink, + extractRpcId, + NULL_ENVELOPE_LOG_SINK, + type EnvelopeLogEntry, +} from "../src/envelope-log-sink.ts"; + +/** A temp directory per test. Cleanup is deliberately non-recursive -- + * unlink the one file we created, then rmdir -- so a stray file makes the + * test fail loudly instead of being silently blown away. */ +function withTempDir(run: (dir: string) => void): void { + const dir = mkdtempSync(join(tmpdir(), "acs-envelope-log-")); + try { + run(dir); + } finally { + try { + unlinkSync(join(dir, "envelopes.jsonl")); + } catch { + // the test may not have produced a log at all -- that is the point of some of them + } + rmdirSync(dir); + } +} + +function readEntries(path: string): EnvelopeLogEntry[] { + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as EnvelopeLogEntry); +} + +const REQUEST = { jsonrpc: "2.0", method: "steps/toolCallRequest", id: 7, params: { acs_version: "0.1.0" } }; +const RESPONSE = { jsonrpc: "2.0", id: 7, result: { decision: "deny" } }; + +describe("createEnvelopeLogSink -- the envelope log's JSONL format", () => { + it("writes one line per call, with a monotonic seq starting at 1", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const sink = createEnvelopeLogSink({ path }); + + sink.write("request", REQUEST, "steps/toolCallRequest"); + sink.write("response", RESPONSE, "steps/toolCallRequest"); + + const entries = readEntries(path); + expect(entries.map((e) => e.seq)).toEqual([1, 2]); + expect(entries.map((e) => e.direction)).toEqual(["request", "response"]); + }); + }); + + // Checks that the JSON value reaches the envelope log unmodified -- + // nothing stripped, nothing reordered. This is not byte identity with the + // wire: the sink is handed the already-parsed result of `await + // req.json()`. + it("records the envelope unmodified -- no reformatting or stripping", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + createEnvelopeLogSink({ path }).write("request", REQUEST, "steps/toolCallRequest"); + + expect(readEntries(path)[0]?.envelope).toEqual(REQUEST); + }); + }); + + it("carries rpc_id from both directions, so the Inspector can pair them (P4)", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const sink = createEnvelopeLogSink({ path }); + + sink.write("request", REQUEST, "steps/toolCallRequest"); + sink.write("response", RESPONSE, "steps/toolCallRequest"); + + expect(readEntries(path).map((e) => e.rpc_id)).toEqual([7, 7]); + }); + }); + + it("stamps recorded_at from the injected clock", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const sink = createEnvelopeLogSink({ path, now: () => new Date("2026-08-09T12:04:31.221Z") }); + + sink.write("request", REQUEST, "steps/toolCallRequest"); + + expect(readEntries(path)[0]?.recorded_at).toBe("2026-08-09T12:04:31.221Z"); + }); + }); + + it("records method as null when the caller cannot determine one", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + createEnvelopeLogSink({ path }).write("response", { jsonrpc: "2.0", id: null, error: { code: -32700 } }, null); + + const entry = readEntries(path)[0]; + expect(entry?.method).toBeNull(); + expect(entry?.rpc_id).toBeNull(); + }); + }); + + // The whole reason the sink is a module rather than three inline + // appendFileSync calls. + it("never throws when the log path is unwritable, reports once, and goes quiet", () => { + withTempDir((dir) => { + const blocker = join(dir, "envelopes.jsonl"); + writeFileSync(blocker, ""); + // A path *through* a regular file: mkdirSync and appendFileSync both + // fail with ENOTDIR, deterministically, on every platform. + const path = join(blocker, "nested", "envelopes.jsonl"); + const errors: unknown[] = []; + const sink = createEnvelopeLogSink({ path, onError: (error) => errors.push(error) }); + + expect(() => sink.write("request", REQUEST, "steps/toolCallRequest")).not.toThrow(); + expect(() => sink.write("response", RESPONSE, "steps/toolCallRequest")).not.toThrow(); + expect(errors.length).toBe(1); + }); + }); + + it("never throws on an envelope JSON.stringify cannot serialize", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const errors: unknown[] = []; + const sink = createEnvelopeLogSink({ path, onError: (error) => errors.push(error) }); + const circular: Record = { id: 1 }; + circular.self = circular; + + expect(() => sink.write("request", circular, "steps/toolCallRequest")).not.toThrow(); + expect(errors.length).toBe(1); + }); + }); + + it("never throws when onError itself throws at construction time", () => { + withTempDir((dir) => { + const blocker = join(dir, "envelopes.jsonl"); + writeFileSync(blocker, ""); + const path = join(blocker, "nested", "envelopes.jsonl"); + + expect(() => { + createEnvelopeLogSink({ + path, + onError: () => { + throw new Error("onError threw"); + }, + }); + }).not.toThrow(); + }); + }); + + it("never throws when onError itself throws at write time, and disables the sink", () => { + withTempDir((dir) => { + const path = join(dir, "envelopes.jsonl"); + const blocker = join(dir, "envelopes.jsonl"); + writeFileSync(blocker, ""); + + const sink = createEnvelopeLogSink({ + path: join(blocker, "nested", "envelopes.jsonl"), + onError: () => { + throw new Error("onError threw"); + }, + }); + + expect(() => sink.write("request", REQUEST, "steps/toolCallRequest")).not.toThrow(); + // The sink should be disabled, so the second write is a silent no-op + expect(() => sink.write("response", RESPONSE, "steps/toolCallRequest")).not.toThrow(); + }); + }); + + it("NULL_ENVELOPE_LOG_SINK writes nothing and never throws", () => { + expect(() => NULL_ENVELOPE_LOG_SINK.write("request", REQUEST, "steps/toolCallRequest")).not.toThrow(); + expect(NULL_ENVELOPE_LOG_SINK.path).toBeNull(); + }); +}); + +describe("extractRpcId", () => { + it("reads string and number ids", () => { + expect(extractRpcId({ id: 7 })).toBe(7); + expect(extractRpcId({ id: "abc" })).toBe("abc"); + }); + + it("returns null for a missing, null, or non-scalar id", () => { + expect(extractRpcId({})).toBeNull(); + expect(extractRpcId({ id: null })).toBeNull(); + expect(extractRpcId({ id: { nested: true } })).toBeNull(); + expect(extractRpcId("not an object")).toBeNull(); + }); +}); diff --git a/packages/guardian/test/server.test.ts b/packages/guardian/test/server.test.ts index 202f2c9a..33791346 100644 --- a/packages/guardian/test/server.test.ts +++ b/packages/guardian/test/server.test.ts @@ -1,14 +1,17 @@ import { describe, expect, it, beforeAll, afterAll } from "bun:test"; -import { readFileSync } from "node:fs"; +import { copyFileSync, mkdirSync, readFileSync, readdirSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import { join } from "node:path"; import Ajv2020 from "ajv/dist/2020.js"; import addFormats from "ajv-formats"; import { startGuardian } from "../src/index.ts"; +import { toRepoRelativeMessage } from "../src/server.ts"; const HANDSHAKE_SCHEMA_PATH = "spec/acs/specification/v0.1.0/handshake.json"; /** Compiles the ServerHello $def straight out of the pinned handshake.json -- * not a hand-copied shape -- so this test fails the moment our ServerHello - * drifts from the schema, per the task's "read the schema yourself" note. */ + * drifts from the schema. */ function validateServerHello(candidate: unknown): void { const handshakeSchema = JSON.parse(readFileSync(HANDSHAKE_SCHEMA_PATH, "utf8")) as { $defs: { ServerHello: Record }; @@ -144,18 +147,17 @@ describe("startGuardian POST /acs", () => { }); }); -// Fix wave finding 1 -- a real fail-open bug: an unhandled throw from -// assemblePreToolCallSnapshot/bridge.evaluate/mapVerdict inside handleAcsRequest used -// to escape uncaught, and Bun.serve's default error page for a rejected -// fetch() is `text/html`, not JSON. guardianClient.post's `res.json()` would -// then throw a SyntaxError instead of surfacing a JSON-RPC error, and -// acs-hook.ts's catch-all exits 1 with nothing on stdout -- Claude Code -// treats that as "the hook never fired" and the tool call proceeds -// ungoverned. This guards the fix, against a real (not mocked) AGT -// evaluation -- only mapping.yaml is swapped for a fixture that marks -// `allow` require_policy_references, so a genuine AGT "allow" verdict for a -// benign command (which carries no reason/message) makes mapVerdict throw -// inside handleAcsRequest for real. +// An unhandled throw from assemblePreToolCallSnapshot, bridge.evaluate, or +// mapVerdict inside handleAcsRequest must never escape uncaught: Bun.serve's +// default error page for a rejected fetch() is `text/html`, not JSON, so +// guardianClient.post's `res.json()` would throw a SyntaxError instead of +// surfacing a JSON-RPC error, acs-hook.ts's catch-all would exit 1 with +// nothing on stdout, and Claude Code would read that as "the hook never +// fired" and let the tool call proceed ungoverned. This test exercises that +// guard against a real (not mocked) AGT evaluation: only mapping.yaml is +// swapped for a fixture that marks `allow` require_policy_references, so a +// genuine AGT "allow" verdict for a benign command (which carries no +// reason/message) makes mapVerdict throw inside handleAcsRequest for real. describe("startGuardian POST /acs -- evaluation failure inside handleAcsRequest", () => { it("a real mapVerdict throw (require_policy_references unmet) still returns a parseable JSON-RPC error in -32000..-32099, not an HTML 500", async () => { const guardian = await startGuardian({ @@ -184,10 +186,519 @@ describe("startGuardian POST /acs -- evaluation failure inside handleAcsRequest" }); }); -// PR #10 review, Critical: mapping.yaml's intervention_points table is what -// V7's conformance matrix publishes, and the runtime used to hardcode -// "pre_tool_call" instead of consulting it, so the two could disagree without -// anything failing. +const REPO_ROOT = fileURLToPath(new URL("../../../", import.meta.url)); +const GUARDIAN_PKG = join(REPO_ROOT, "packages", "guardian"); + +/** + * A stable, predictable name rather than an `mkdtempSync` random one. This + * tree has to live *inside* `packages/guardian/` -- not under a repo-wide + * temp directory -- because it is a relative-path trick: + * `validate-envelope.ts` resolves its schema root three directories + * up from its own `import.meta.url`, so the copy has to sit at the same + * depth under `packages/guardian/` for that resolution to land one level + * short, on purpose (see the doc comment below). Bun's workspace module + * resolution for the copy's own `import`s (`agt-bridge`, etc.) also + * resolves relative to where the copy physically sits, which only works + * predictably inside the real package tree. + * + * A fixed name means a killed run's leftover copy is not a fresh, unignored + * directory tsc has never heard of: it is *this* directory, already covered + * by `.gitignore` and by `tsconfig.json`'s `exclude`, so it cannot make + * `bun run typecheck` see a stray duplicate of the Guardian's own source. A + * run that starts while a previous killed run's copy is still here fails + * loudly (`mkdirSync` on an existing directory throws) rather than quietly + * reusing stale files -- the cure for that is deleting the leftover + * directory by hand, not adding recovery logic that would have to guess + * whether stale contents are safe to remove. + */ +const SCHEMALESS_SCRATCH_DIR = join(GUARDIAN_PKG, "tmp-schemaless-scratch"); + +/** + * Runs `body` against a Guardian whose `validate-envelope.ts` cannot find the + * ACS schemas -- the tree-cloned-without-`--recurse-submodules` case, which + * is exactly the scenario handleAcsRequest's outer net exists to catch. + * + * It is reproduced by *relocation*, not by mocking and not by touching + * `spec/`. `validate-envelope.ts` derives SCHEMA_ROOT from its own + * `import.meta.url` as `../../../spec/acs/specification/...`, so an identical + * copy of `packages/guardian/src` placed one directory deeper resolves that + * path to `packages/spec/acs/...`, which does not exist. Every line of + * Guardian code that then runs is the real, current source -- the files are + * copied at test time, so they cannot drift from `src/` -- and the failure it + * produces is a real ENOENT out of `readdirSync`, thrown at request time + * because `buildAjv()` is lazy. The first call to this function gets its own + * module instance, so it cannot poison the Ajv registry the rest of this + * suite shares. A later call importing the same fixed + * `SCHEMALESS_SCRATCH_DIR` path a second time does not get a second fresh + * instance -- Bun's module cache keys by resolved path, so it gets back the + * *first* call's already-loaded module, and the freshly copied files on + * disk for that later call go unread. Benign here (every call copies + * byte-identical source, and this suite's own Ajv registry is never shared + * with the copy either way), but worth being precise about now that the + * directory name is fixed rather than fresh per call. + * + * Deletions here are explicit per file (repo constraint: nothing recursive). + */ +async function withSchemalessGuardian( + body: (guardian: { url: string; logPath: string }) => Promise, +): Promise { + const root = SCHEMALESS_SCRATCH_DIR; + const srcDir = join(root, "src"); + const logPath = join(root, "envelopes.jsonl"); + const copied = readdirSync(join(GUARDIAN_PKG, "src")).filter((f) => f.endsWith(".ts")); + let guardian: { close(): Promise } | undefined; + + // Everything after this mkdirSync is inside the try: a failure while + // copying, importing, or booting would otherwise leave a directory of + // stray .ts files sitting inside packages/guardian. + mkdirSync(root); + try { + mkdirSync(srcDir); + for (const file of copied) { + copyFileSync(join(GUARDIAN_PKG, "src", file), join(srcDir, file)); + } + + const relocated = (await import(join(srcDir, "index.ts"))) as typeof import("../src/index.ts"); + const started = await relocated.startGuardian({ + port: 0, + manifestPath: join(REPO_ROOT, "policy", "manifest.yaml"), + mappingPath: join(REPO_ROOT, "mapping.yaml"), + envelopeLogPath: logPath, + }); + guardian = started; + + await body({ url: started.url, logPath }); + } finally { + await guardian?.close(); + for (const path of [logPath, ...copied.map((file) => join(srcDir, file))]) { + try { + unlinkSync(path); + } catch { + // a run that failed early never created every one of these + } + } + for (const dir of [srcDir, root]) { + try { + rmdirSync(dir); + } catch { + // same + } + } + } +} + +/** + * One scratch directory *per distinct fake source*, not shared the way + * `SCHEMALESS_SCRATCH_DIR` is shared across `withSchemalessGuardian`'s four + * tests. Those four all copy the *same* real files every time, so whichever + * call's module instance Bun's cache happens to answer with behaves + * identically. These two fake sources differ from each other, and Bun's + * module cache keys by resolved path: reusing one directory for both would + * make the second call's `import()` return the *first* call's + * already-loaded module, silently exercising the wrong test double. Two + * names, so each call gets a path Bun has never loaded before. + */ +const UNDEFINED_MESSAGE_SCRATCH_DIR = join(GUARDIAN_PKG, "tmp-undefined-message-scratch"); +const THROWING_MESSAGE_ACCESSOR_SCRATCH_DIR = join(GUARDIAN_PKG, "tmp-throwing-message-accessor-scratch"); + +/** + * A test double, not the real `validate-envelope.ts`. It exists to force a + * real `Error` whose `.message` has been overwritten to `undefined` through + * `dispatch`'s one rethrow route -- the same route `withSchemalessGuardian` + * above uses for a real ENOENT, but that route cannot also produce a + * non-string `.message`: nothing in the real schema-validation path does + * that to an error it throws. `EnvelopeValidationError` is redeclared here, + * distinct from the real one, so `dispatch`'s + * `error instanceof EnvelopeValidationError` check -- reading *this* file's + * class, inside the relocated copy -- correctly comes back `false` and + * rethrows, the same way it would for any error the real module didn't + * throw as an `EnvelopeValidationError`. + */ +const UNDEFINED_MESSAGE_VALIDATE_ENVELOPE_SOURCE = ` +export class EnvelopeValidationError extends Error {} + +export function validateEnvelope(_input) { + const error = new Error("this message is about to be erased"); + error.message = undefined; + throw error; +} + +// Must track validate-envelope.ts's real export surface, not just the two +// symbols this double overrides: server.ts imports isToolCallRequest from the +// same module, so a double that omits it fails to import rather than +// exercising the pathological throw these tests exist for. Mirrors the real +// narrowing exactly -- it is unreachable here (validateEnvelope always +// throws) but a double that lies about behaviour is worse than one that +// does not compile. +export function isToolCallRequest(envelope) { + return envelope.method === "steps/toolCallRequest"; +} +`; + +/** + * A second test double, covering the case where `toRepoRelativeMessage`'s + * own `error instanceof Error ? error.message : error` line can itself + * throw: `.message` as an accessor that throws on get -- which the plain + * `undefined`-message double above does not exercise, since overwriting + * `.message` with a value never triggers a getter. `EnvelopeValidationError` + * is redeclared here for the same reason as the double above. + * + * Deliberately *not* a getPrototypeOf-trapping Proxy. That shape is real and + * is covered directly, at the unit level, + * below -- but it cannot reach `toRepoRelativeMessage` unmutated through + * this route: `dispatch`'s own `error instanceof EnvelopeValidationError` + * check runs first, and `instanceof` needs exactly the trapped + * `[[GetPrototypeOf]]` internal method to walk the prototype chain, so the + * *trap's own thrown Error* replaces the Proxy at that point -- a normal, + * well-behaved Error reaches the outer catch instead, and the interesting + * case never arrives. A throwing `.message` accessor has no such problem: + * `instanceof` never touches `.message`, so a real `Error` carrying one + * passes through dispatch's check untouched and reaches + * `toRepoRelativeMessage` exactly as thrown. + */ +const THROWING_MESSAGE_ACCESSOR_VALIDATE_ENVELOPE_SOURCE = ` +export class EnvelopeValidationError extends Error {} + +export function validateEnvelope(_input) { + const error = new Error("real message, about to be hidden behind a throwing getter"); + Object.defineProperty(error, "message", { + get() { + throw new Error("message getter blew up"); + }, + }); + throw error; +} + +// Must track validate-envelope.ts's real export surface, not just the two +// symbols this double overrides: server.ts imports isToolCallRequest from the +// same module, so a double that omits it fails to import rather than +// exercising the pathological throw these tests exist for. Mirrors the real +// narrowing exactly -- it is unreachable here (validateEnvelope always +// throws) but a double that lies about behaviour is worse than one that +// does not compile. +export function isToolCallRequest(envelope) { + return envelope.method === "steps/toolCallRequest"; +} +`; + +/** + * Runs \`body\` against a Guardian whose \`validate-envelope.ts\` has been + * replaced by \`fakeSource\` -- reproducing, through \`dispatch\`'s one + * rethrow route, a pathological value that a real \`validateEnvelope\` + * would never throw. Structurally identical to \`withSchemalessGuardian\`: a + * real, unmodified copy of every other file in \`packages/guardian/src\`, + * dynamically imported from its own scratch directory so it is a distinct + * module instance, with only \`validate-envelope.ts\` swapped for the + * double. No \`envelopeLogPath\` -- these tests need no envelope log, and + * \`NULL_ENVELOPE_LOG_SINK\`'s totality is already covered elsewhere. + * + * \`root\` is the caller's -- one of the two scratch-dir constants above, + * never shared between two different \`fakeSource\`s (see their doc comment + * for why that matters here specifically). + */ +async function withFakeValidateEnvelopeGuardian( + root: string, + fakeSource: string, + body: (guardian: { url: string }) => Promise, +): Promise { + const srcDir = join(root, "src"); + const copied = readdirSync(join(GUARDIAN_PKG, "src")).filter((f) => f.endsWith(".ts") && f !== "validate-envelope.ts"); + let guardian: { close(): Promise } | undefined; + + mkdirSync(root); + try { + mkdirSync(srcDir); + for (const file of copied) { + copyFileSync(join(GUARDIAN_PKG, "src", file), join(srcDir, file)); + } + writeFileSync(join(srcDir, "validate-envelope.ts"), fakeSource); + + const relocated = (await import(join(srcDir, "index.ts"))) as typeof import("../src/index.ts"); + const started = await relocated.startGuardian({ + port: 0, + manifestPath: join(REPO_ROOT, "policy", "manifest.yaml"), + mappingPath: join(REPO_ROOT, "mapping.yaml"), + }); + guardian = started; + + await body({ url: started.url }); + } finally { + await guardian?.close(); + for (const path of [join(srcDir, "validate-envelope.ts"), ...copied.map((file) => join(srcDir, file))]) { + try { + unlinkSync(path); + } catch { + // a run that failed early never created every one of these + } + } + for (const dir of [srcDir, root]) { + try { + rmdirSync(dir); + } catch { + // same + } + } + } +} + +// `dispatch` rethrows any non-EnvelopeValidationError, and the outer net in +// handleAcsRequest is what catches it: an uncaught rethrow would leave +// Bun.serve answering with its default `text/html` 500, guardian-client's +// unconditional `res.json()` would throw `JSON Parse error: Unrecognized +// token '<'`, acs-hook.ts's catch-all would exit 1 with empty stdout, and +// Claude Code would read that as "the hook didn't fire" and let the tool +// call proceed ungoverned. Without this net, the envelope log would also +// record only the request, leaving the Inspector unable to show that a +// response was ever sent. +describe("startGuardian POST /acs -- the outer net around dispatch", () => { + it("answers a throw from validateEnvelope itself with parseable JSON-RPC in -32000..-32099, never an HTML 500", async () => { + await withSchemalessGuardian(async ({ url }) => { + const res = await fetch(url, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(toolCallEnvelope("ls -la", { id: 5 })), + }); + + // Read as text first and parse by hand, so a regression reports the + // HTML body it actually got instead of an opaque SyntaxError from + // res.json() -- which is precisely what guardian-client would throw. + const text = await res.text(); + expect(text.slice(0, 1)).toBe("{"); + const response = JSON.parse(text) as JsonRpcResponse; + + expect(response.jsonrpc).toBe("2.0"); + expect(response.id).toBe(5); + expect(response.error).toBeDefined(); + expect(response.error?.code).toBeGreaterThanOrEqual(-32099); + expect(response.error?.code).toBeLessThanOrEqual(-32000); + }); + }); + + // The real ENOENT `withSchemalessGuardian` provokes names this machine's + // absolute path in full (`readdirSync` on a schema directory that does + // not exist at the relocated copy's resolved path). toRepoRelativeMessage + // strips only the repo-root prefix from it, so the diagnostic remainder + // (the ENOENT text and the repo-relative path) is still there for a real + // reader to use, without disclosing where this tree sits on disk. + it("strips this repo's absolute root out of a real error message before it reaches the client", async () => { + await withSchemalessGuardian(async ({ url }) => { + const response = await postAcs(url, toolCallEnvelope("ls -la")); + + expect(response.error).toBeDefined(); + const message = response.error?.message ?? ""; + expect(message).not.toContain(REPO_ROOT); + expect(message).toContain("ENOENT"); + // The diagnostic remainder: which schema directory was missing, + // relative rather than absolute. (Relative to the *relocated* copy's + // own root, one level shallower than this file's REPO_ROOT above -- + // see withSchemalessGuardian's doc comment -- so no "packages/" + // prefix here; that is this test harness's relocation depth, not a + // second absolute-path leak.) + expect(message).toContain("spec/acs/specification/v0.1.0"); + }); + }); + + it("carries no decision -- a Guardian-side failure is an error here, not a synthesized deny", async () => { + await withSchemalessGuardian(async ({ url }) => { + const response = await postAcs(url, toolCallEnvelope("rm -rf /")); + + expect(response.result).toBeUndefined(); + expect((response as Record).decision).toBeUndefined(); + expect(JSON.stringify(response)).not.toContain("deny"); + }); + }); + + it("records both the request and the response, so the envelope log has no unrecorded exit", async () => { + await withSchemalessGuardian(async ({ url, logPath }) => { + await postAcs(url, toolCallEnvelope("ls -la", { id: 11 })); + + const lines = readFileSync(logPath, "utf8").trim().split("\n"); + const entries = lines.map((line) => JSON.parse(line) as { direction: string; rpc_id: unknown }); + expect(entries.map((e) => e.direction)).toEqual(["request", "response"]); + // Paired by JSON-RPC id, which is what lets the Inspector show the + // failure beside the request that caused it. + expect(entries.map((e) => e.rpc_id)).toEqual([11, 11]); + }); + }); + + // `instanceof Error` does not guarantee `.message` is a string -- true of + // the real ENOENT the test above forces, but not something + // toRepoRelativeMessage can assume in general. An Error whose `.message` + // has been overwritten to `undefined` would throw `TypeError: undefined + // is not an object (evaluating 'message.replace')` out of the helper + // itself if it made that assumption -- and unlike the inner catch's own + // throw (contained by this outer catch), a throw *from* the outer catch + // has nothing above `handleAcsRequest` to catch it: Bun.serve's fetch + // handler has no try, so it would answer with the unrecorded HTML 500 the + // module header exists to prevent. + it("does not let a real Error with a non-string .message escape the outer catch as an HTML 500", async () => { + await withFakeValidateEnvelopeGuardian(UNDEFINED_MESSAGE_SCRATCH_DIR, UNDEFINED_MESSAGE_VALIDATE_ENVELOPE_SOURCE, async ({ url }) => { + const res = await fetch(url, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(toolCallEnvelope("ls -la", { id: 9 })), + }); + + const text = await res.text(); + expect(text.slice(0, 1)).toBe("{"); + const response = JSON.parse(text) as JsonRpcResponse; + + expect(response.jsonrpc).toBe("2.0"); + expect(response.id).toBe(9); + expect(response.error).toBeDefined(); + expect(response.error?.code).toBeGreaterThanOrEqual(-32099); + expect(response.error?.code).toBeLessThanOrEqual(-32000); + // Total, coerced to text, rather than thrown. + expect(response.error?.message).toContain("undefined"); + }); + }); + + // An Error whose `.message` is an accessor that throws on get defeats + // `error instanceof Error ? error.message : error` inside + // toRepoRelativeMessage itself. Unreachable from any real throw site in + // this repo today -- belt and braces, not a reaction to a live bug (see + // toRepoRelativeMessage's doc comment) -- but the unit assertions in the + // describe block below only prove the helper itself is total; this + // proves the outer net around it still holds when the value it's handed + // is this pathological. (A getPrototypeOf-trapping Proxy is covered at + // the unit level only, not here -- see + // THROWING_MESSAGE_ACCESSOR_VALIDATE_ENVELOPE_SOURCE's doc comment for + // why that one specifically cannot reach toRepoRelativeMessage unmutated + // through dispatch's rethrow route.) + it("does not let an Error with a throwing .message accessor escape the outer catch as an HTML 500 either", async () => { + await withFakeValidateEnvelopeGuardian( + THROWING_MESSAGE_ACCESSOR_SCRATCH_DIR, + THROWING_MESSAGE_ACCESSOR_VALIDATE_ENVELOPE_SOURCE, + async ({ url }) => { + const res = await fetch(url, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(toolCallEnvelope("ls -la", { id: 10 })), + }); + + const text = await res.text(); + expect(text.slice(0, 1)).toBe("{"); + const response = JSON.parse(text) as JsonRpcResponse; + + expect(response.jsonrpc).toBe("2.0"); + expect(response.id).toBe(10); + expect(response.error).toBeDefined(); + expect(response.error?.code).toBeGreaterThanOrEqual(-32099); + expect(response.error?.code).toBeLessThanOrEqual(-32000); + expect(response.error?.message).toContain(""); + }, + ); + }); +}); + +describe("toRepoRelativeMessage", () => { + // `instanceof Error` says nothing about what `.message` was reassigned to + // after construction, so this function must not assume it is a string. + // Exercised directly (rather than only through + // withFakeValidateEnvelopeGuardian's HTTP round trip) so every shape of + // `unknown` a catch clause can hand it is covered without standing up a + // Guardian for each one. + it("never throws, for an Error whose .message is not a string", () => { + const undefinedMessage = new Error("erased below"); + (undefinedMessage as { message: unknown }).message = undefined; + const numberMessage = new Error("erased below"); + (numberMessage as { message: unknown }).message = 42; + const objectMessage = new Error("erased below"); + (objectMessage as { message: unknown }).message = { nested: true }; + + expect(toRepoRelativeMessage(undefinedMessage)).toBe("undefined"); + expect(toRepoRelativeMessage(numberMessage)).toBe("42"); + expect(toRepoRelativeMessage(objectMessage)).toBe("[object Object]"); + }); + + it("is total for non-Error unknown values too, matching what a catch clause can hand it", () => { + expect(toRepoRelativeMessage("a plain string")).toBe("a plain string"); + expect(toRepoRelativeMessage(42)).toBe("42"); + expect(toRepoRelativeMessage(null)).toBe("null"); + expect(toRepoRelativeMessage(undefined)).toBe("undefined"); + expect(toRepoRelativeMessage({ some: "object" })).toBe("[object Object]"); + }); + + // The contract this function exists to uphold is that nothing escapes the + // outer net, and "unreachable today" should not be load-bearing for that + // (see this function's doc comment). Four shapes, each defeating a + // different step of `String(error instanceof Error ? error.message : + // error)`: + // - an Error whose `.message` is a throwing accessor + // - a value whose `toString`/`valueOf` both throw, so `String()` itself + // throws on the non-Error branch + // - a Proxy that throws on `get` (String() needs to read + // Symbol.toPrimitive/toString/valueOf off it) + // - a Proxy that throws on `getPrototypeOf`, defeating `instanceof + // Error` before `String()` is ever reached at all + it("never throws, even for values that defeat message access, stringification, or property/prototype traps", () => { + const throwingAccessor = new Error("real message, about to be hidden behind a throwing getter"); + Object.defineProperty(throwingAccessor, "message", { + get() { + throw new Error("message getter blew up"); + }, + }); + + const throwingToString = { + toString() { + throw new Error("toString blew up"); + }, + valueOf() { + throw new Error("valueOf blew up"); + }, + }; + + const throwingGetProxy = new Proxy( + {}, + { + get() { + throw new Error("get trap blew up"); + }, + }, + ); + + const throwingGetPrototypeOfProxy = new Proxy( + {}, + { + getPrototypeOf() { + throw new Error("getPrototypeOf trap blew up"); + }, + }, + ); + + expect(toRepoRelativeMessage(throwingAccessor)).toBe(""); + expect(toRepoRelativeMessage(throwingToString)).toBe(""); + expect(toRepoRelativeMessage(throwingGetProxy)).toBe(""); + expect(toRepoRelativeMessage(throwingGetPrototypeOfProxy)).toBe(""); + }); + + // This file's own REPO_ROOT keeps the trailing slash `fileURLToPath` + // gives a directory URL -- fine for join()ing against, but these two + // tests need the bare root, with nothing after it, to build "root + + // separator + subpath" and "root + suffix" strings without accidentally + // doubling or misplacing a slash. + const REPO_ROOT_BARE = REPO_ROOT.replace(/[/\\]+$/, ""); + + it("strips this repo's root, with or without a trailing separator", () => { + expect(toRepoRelativeMessage(new Error(`${REPO_ROOT_BARE}/packages/spec/acs`))).toBe("packages/spec/acs"); + expect(toRepoRelativeMessage(new Error(REPO_ROOT_BARE))).toBe(""); + }); + + // An un-anchored match on REPO_ROOT as a bare prefix would also strip a + // *sibling* directory whose name merely extends the root (a `_old` backup + // clone, say) -- not a disclosure of this tree's own location, since it + // names a different directory entirely, but a misleading diagnostic that + // would then read as if it were a path under this repo. + it("leaves a sibling directory whose name extends the repo root untouched", () => { + const siblingPath = `${REPO_ROOT_BARE}_old/packages/spec`; + + expect(toRepoRelativeMessage(new Error(siblingPath))).toBe(siblingPath); + }); +}); + +// The intervention point comes from mapping.yaml's own `intervention_points` +// table rather than from a hardcoded "pre_tool_call", so the table cannot +// drift away from what the runtime actually does without a test catching it. describe("startGuardian POST /acs -- the intervention point comes from mapping.yaml", () => { it("evaluates the point the table names, not pre_tool_call: a moved row changes the decision", async () => { // The fixture answers steps/toolCallRequest with `output`, which diff --git a/packages/inspector/package.json b/packages/inspector/package.json new file mode 100644 index 00000000..4744b90f --- /dev/null +++ b/packages/inspector/package.json @@ -0,0 +1,8 @@ +{ + "name": "inspector", + "private": true, + "type": "module", + "version": "0.1.0", + "main": "src/index.ts", + "types": "src/index.ts" +} diff --git a/packages/inspector/src/index.ts b/packages/inspector/src/index.ts new file mode 100644 index 00000000..fd74c909 --- /dev/null +++ b/packages/inspector/src/index.ts @@ -0,0 +1,19 @@ +/** Public surface of the inspector package. */ +export { + tailEnvelopeLog, + type EnvelopeLogDirection, + type EnvelopeLogEntry, + type TailEnvelopeLogOptions, +} from "./tail-envelope-log.ts"; +export { + outcomeMessageOf, + renderDecisionBadge, + renderEnvelopeLogEntry, + renderOutcome, + renderRpcError, + type DecisionMessage, + type OutcomeMessage, + type PolicyReference, + type RenderOptions, + type RpcErrorMessage, +} from "./render.ts"; diff --git a/packages/inspector/src/main.ts b/packages/inspector/src/main.ts new file mode 100644 index 00000000..401735ac --- /dev/null +++ b/packages/inspector/src/main.ts @@ -0,0 +1,45 @@ +/** + * The Envelope Inspector's entrypoint -- `bun run inspector`. + * + * A third terminal beside `bun run guardian` and the agent host: it tails + * the envelope log and prints each ACS envelope as it crosses the wire. Not + * re-exported from ./index.ts -- this is a process entrypoint, not a + * library call. + * + * `ACS_ENVELOPE_LOG` defaults to `.acs/envelopes.jsonl`, the same default + * packages/guardian/src/main.ts writes to, so the two agree without either + * hardcoding the other's value. + */ +import { tailEnvelopeLog } from "./tail-envelope-log.ts"; +import { renderEnvelopeLogEntry } from "./render.ts"; + +const DEFAULT_ENVELOPE_LOG = ".acs/envelopes.jsonl"; + +const argv = process.argv.slice(2); +const fromStart = argv.includes("--from-start"); +// `--envelope-log`, not `--path`: this stream is the envelope log, and a +// generic `--path` would be the flag for whichever stream happened to come +// first, leaving any sibling stream to carry the qualifier. Named for its +// own artifact, so the flag rhymes with the env var that overrides the same +// thing. +const envelopeLogFlag = argv.indexOf("--envelope-log"); +const flagValue = envelopeLogFlag === -1 ? undefined : argv[envelopeLogFlag + 1]; + +if (envelopeLogFlag !== -1 && (flagValue === undefined || flagValue.startsWith("--"))) { + console.error("usage: bun run inspector -- [--from-start] [--envelope-log ]"); + process.exit(2); +} + +const path = flagValue ?? process.env.ACS_ENVELOPE_LOG ?? DEFAULT_ENVELOPE_LOG; +const color = Boolean(process.stdout.isTTY) && !process.env.NO_COLOR; + +const controller = new AbortController(); +process.on("SIGINT", () => controller.abort()); + +console.log(`Envelope Inspector — tailing ${path}${fromStart ? " (from the start)" : ""}`); +console.log("Ctrl-C to stop.\n"); + +for await (const entry of tailEnvelopeLog({ path, fromStart, signal: controller.signal })) { + console.log(renderEnvelopeLogEntry(entry, { color })); + console.log(""); +} diff --git a/packages/inspector/src/render.ts b/packages/inspector/src/render.ts new file mode 100644 index 00000000..16614a9b --- /dev/null +++ b/packages/inspector/src/render.ts @@ -0,0 +1,241 @@ +/** + * Renders the two things the Inspector prints: the envelope stream and the + * ACS decision badge. + * + * Every function here is pure: no clock, no env, no process. The CLI decides + * whether the terminal wants ANSI and passes `color`; tests assert exact + * plain strings. The badge reads only ACS's own `decision`, `reason_codes`, + * and `policy_references` fields -- nothing here knows what policy runtime + * produced a decision. + * + * `renderDecisionBadge` is told a decision rather than handed a log row to + * interrogate: `outcomeMessageOf` is the one place that reads an envelope's + * shape, translating a log line into the small message the renderers need, + * so a caller can build and render that message without owning an envelope + * log at all. + * + * A decision and an error are two outcomes, not two decisions: a + * schema-invalid envelope comes back as a JSON-RPC error, which is + * deliberately not the same thing as an ACS `deny`. `OutcomeMessage` is a + * discriminated union over the two, with its own renderer per arm -- + * `renderDecisionBadge` for a decision, `renderRpcError` for a response that + * carried none -- and `renderOutcome` dispatches between them for the + * stream renderer. + */ +import type { EnvelopeLogEntry } from "./tail-envelope-log.ts"; + +export type RenderOptions = { color?: boolean; indent?: number }; + +const RESET = "\u001b[0m"; +const RED = "\u001b[31m"; +const GREEN = "\u001b[32m"; +const YELLOW = "\u001b[33m"; +const CYAN = "\u001b[36m"; +const DIM = "\u001b[2m"; + +export type PolicyReference = { policy_id?: string; policy_version?: string; rule_id?: string }; + +/** + * What a caller tells the decision badge, already narrowed to the ACS + * fields it renders. Only ever an actual ACS decision -- see + * `OutcomeMessage` for why this type covers only decisions. + */ +export type DecisionMessage = { + decision: string; + reason_codes: string[]; + policy_references: PolicyReference[]; +}; + +/** What a response carried instead of a decision: the JSON-RPC error object. */ +export type RpcErrorMessage = { code: number | null; message: string }; + +/** + * What one envelope-log response line reports about its step: a decision, + * or the JSON-RPC error that stood in place of one. + * + * A discriminated union rather than one object with an optional `error` beside + * a `decision`, because a response carries exactly one of them and the other + * arm's fields would have to be invented. A `steps/*` response either names an + * ACS `decision` or is a JSON-RPC error; a caller cannot report both, and + * cannot report neither. + * + * `kind` rather than structural narrowing on a key's presence, so the two arms + * stay tellable apart by a caller that does not already know which fields + * belong to which -- and so the union reads as two outcomes rather than as a + * decision with an error mode. + */ +export type OutcomeMessage = + | ({ kind: "decision" } & DecisionMessage) + | ({ kind: "error" } & RpcErrorMessage); + +type DecisionResult = { + decision?: unknown; + reason_codes?: unknown; + policy_references?: unknown; +}; +type ResponseEnvelope = { result?: DecisionResult; error?: { code?: unknown; message?: unknown } }; + +function paint(text: string, color: string, enabled: boolean): string { + return enabled ? `${color}${text}${RESET}` : text; +} + +/** `2026-08-09T12:04:31.221Z` -> `12:04:31.221`. Sliced, not parsed: UTC and + * locale-independent, so rendered output is the same everywhere. */ +function clockOf(recordedAt: string): string { + const time = recordedAt.slice(11, 23); + return time.length === 12 ? time : recordedAt; +} + +function stringList(value: unknown): string[] { + return Array.isArray(value) ? value.filter((item): item is string => typeof item === "string") : []; +} + +/** The `policy_references` a response carried, narrowed to objects. Kept + * separate from the formatting below so the message stays structured: a + * caller building a `DecisionMessage` by hand passes references, not + * pre-rendered strings. */ +function policyReferenceList(value: unknown): PolicyReference[] { + if (!Array.isArray(value)) { + return []; + } + return value.filter((item): item is PolicyReference => typeof item === "object" && item !== null); +} + +/** Formats each `policy_references` entry as `policy_id#rule_id`, falling + * back to the bare `policy_id` when `rule_id` is absent. ACS's schemas do + * not require `rule_id` on a policy_reference, so that fallback is a real + * shape this renders deliberately, not a defect. */ +function formatReferences(references: PolicyReference[]): string[] { + return references.map((ref) => (ref.rule_id ? `${ref.policy_id ?? "?"}#${ref.rule_id}` : `${ref.policy_id ?? "?"}`)); +} + +/** + * What one envelope-log line reports, or null when it reports no outcome at + * all: a request, or a response such as a ServerHello. + * + * The one place in this module that reads an envelope's shape. It is a + * translation, not a collaboration -- it turns an artifact into the message the + * renderers speak -- and confining it here is what lets `renderDecisionBadge` + * be told a decision instead of interrogating a log row for one. + */ +export function outcomeMessageOf(entry: EnvelopeLogEntry): OutcomeMessage | null { + if (entry.direction !== "response") { + return null; + } + const envelope = (typeof entry.envelope === "object" && entry.envelope !== null ? entry.envelope : {}) as ResponseEnvelope; + + if (envelope.error) { + return { + kind: "error", + code: typeof envelope.error.code === "number" ? envelope.error.code : null, + message: typeof envelope.error.message === "string" ? envelope.error.message : "", + }; + } + + const result = envelope.result; + if (!result || typeof result.decision !== "string") { + return null; + } + + return { + kind: "decision", + decision: result.decision, + reason_codes: stringList(result.reason_codes), + policy_references: policyReferenceList(result.policy_references), + }; +} + +/** + * A response that carried no decision, rendered as the error it was. + * + * Its own function rather than an arm of the decision badge: a + * schema-invalid envelope comes back as a JSON-RPC error, which this module + * treats as a distinct outcome from an ACS decision, never as an implicit + * `deny`. The line looks like the badge beside it on purpose, but no type or + * function here calls it a decision. + */ +export function renderRpcError(message: RpcErrorMessage, options: RenderOptions = {}): string { + const color = options.color ?? false; + const code = message.code ?? "?"; + const text = message.message; + return paint(`✖ ERROR ${code}${text ? ` ${text}` : ""}`, RED, color); +} + +/** Renders the ACS decision it is given. */ +export function renderDecisionBadge(message: DecisionMessage, options: RenderOptions = {}): string { + const color = options.color ?? false; + + const reasonCodes = message.reason_codes; + const references = formatReferences(message.policy_references); + + let head: string; + if (message.decision === "deny") { + head = paint("● DENY", RED, color); + } else if (message.decision === "allow" && references.length > 0) { + // A policy fired and the action still proceeded. ACS carries that as + // `allow` with a non-empty `policy_references`, and rendering it + // identically to a clean allow is exactly what this badge exists to + // prevent. + // + // This package carries no policy-runtime vocabulary: the label says only + // what ACS itself reports, not the policy engine's own name for the case. + head = paint("◐ ALLOW (policy fired)", YELLOW, color); + } else if (message.decision === "allow") { + head = paint("○ ALLOW", GREEN, color); + } else { + head = paint(`◆ ${message.decision.toUpperCase()}`, CYAN, color); + } + + // Dimmed rather than left plain: with color:true, painting only `head` + // made a coloured badge read as one coloured half and one plain half. + // `paint` no-ops when `color` is false, so this changes nothing about the + // color:false output the exact-string tests above assert byte-for-byte. + const parts = [head]; + if (reasonCodes.length > 0) { + parts.push(paint(`reason_codes=[${reasonCodes.join(", ")}]`, DIM, color)); + } + if (references.length > 0) { + parts.push(paint(`policy_references=[${references.join(", ")}]`, DIM, color)); + } + return parts.join(" "); +} + +/** The line one envelope-log outcome renders as -- the decision badge for a + * decision, the error line for a response that carried none. One dispatch, + * so the stream renderer does not have to know the arms apart. */ +export function renderOutcome(message: OutcomeMessage, options: RenderOptions = {}): string { + return message.kind === "decision" ? renderDecisionBadge(message, options) : renderRpcError(message, options); +} + +/** + * Header line, optional badge line, then the envelope as pretty JSON. + * + * What the body shows is the JSON value the envelope log recorded, printed + * unmodified: nothing here strips a field, redacts a value, or reorders + * anything. It is not a byte-for-byte replay of the wire -- the Guardian + * records `await req.json()`, so the parse has already collapsed duplicate + * keys, canonicalised number literals (`1.0` -> `1`), and hoisted + * integer-like object keys ahead of the rest, and tool argument names are + * host-controlled, so `arguments` really can carry a key like `"0"`. Storing + * raw bytes instead would make `entry.envelope` a string rather than a JSON + * value, which costs the pretty-printing below and the round-trip contract + * test. + */ +export function renderEnvelopeLogEntry(entry: EnvelopeLogEntry, options: RenderOptions = {}): string { + const color = options.color ?? false; + const arrow = entry.direction === "request" ? "→ REQUEST " : "← RESPONSE"; + const method = entry.method ?? "(no method)"; + const id = entry.rpc_id === null ? "(unpaired)" : `id=${entry.rpc_id}`; + + const header = paint(`── #${entry.seq} ${clockOf(entry.recorded_at)} ${arrow} ${method} ${id}`, DIM, color); + const message = outcomeMessageOf(entry); + const outcome = message === null ? null : renderOutcome(message, options); + // `JSON.stringify` returns `undefined` -- not a string -- for an entry + // whose `envelope` key is absent, and `join` would coerce that to an empty + // line indistinguishable from a real blank body. `isEnvelopeLogEntryShape` + // does not require `envelope` (it is `unknown` by design), so a + // hand-written or truncated envelope-log line can reach here without one. + const body = JSON.stringify(entry.envelope, null, options.indent ?? 2) ?? "(no envelope recorded)"; + + return [header, ...(outcome === null ? [] : [outcome]), body].join("\n"); +} diff --git a/packages/inspector/src/tail-envelope-log.ts b/packages/inspector/src/tail-envelope-log.ts new file mode 100644 index 00000000..7abc01f1 --- /dev/null +++ b/packages/inspector/src/tail-envelope-log.ts @@ -0,0 +1,302 @@ +/** + * tailEnvelopeLog streams the Guardian's JSONL envelope log as it grows, the + * way `tail -f` does. + * + * This package deliberately imports nothing from `guardian` or from any of + * its dependencies. The Inspector reads a file that the Guardian happens to + * write, and holds no compile-time knowledge of the process that produced + * it: envelopes are inspectable on the wire, not through a shared type + * graph. `EnvelopeLogEntry` is therefore re-declared here rather than + * imported. The round-trip test at test/envelope-log-sink-roundtrip.test.ts + * keeps the two declarations in agreement; if they drift, it fails. + * + * The names are the artifact's, not the writer's: `EnvelopeLogEntry` and + * `TailEnvelopeLogOptions` name the log they describe, not the Guardian's + * own writing mechanism, so a second log stream in this package would read + * as this one's sibling rather than force a shared, ambiguous vocabulary + * between them. + * + * Polling rather than fs.watch: appends to a growing file are exactly the + * case where watch semantics differ most across platforms, and a 120ms poll + * on a local demo log costs nothing. + * + * The poll runs on its own timer once started, independent of whether + * anything is actively pulling values from the returned generator. That + * independence is load-bearing, not incidental: a consumer can read one + * entry and then pause before asking for the next one, and the file can be + * truncated and rewritten entirely within that pause. A poll folded into + * the generator's own suspend/resume points only ever inspects the file at + * the instant the consumer resumes it -- by then a truncate-and-rewrite can + * look exactly like ordinary growth (a same-length-or-longer rewrite makes + * the new size come out >= the old offset, so a size-only check never + * fires). A timer that keeps ticking regardless of consumption catches the + * file while it is still sitting at zero. + * + * The timer itself, though, only starts on the first `next()` -- inside the + * returned generator's body, not inside this function. A caller that builds + * a tail and never iterates it (and never aborts) should not leave a timer + * running forever; tying its start to first consumption means an unused + * tail costs nothing. + * + * The starting offset is a separate concern from the timer, and is captured + * synchronously right here, when this function is called -- not lazily + * inside the generator body, which would not run at all until the caller's + * first `next()`. "Start at the current end" has to mean the moment + * `tailEnvelopeLog` was called, not the moment someone first asked it for a + * value; the truncation scenario above still works even though the timer + * itself starts later, because by the time a consumer has read anything at + * all, `next()` has already been called once and the timer is already + * running. + * + * `poll()` runs as a bare timer callback with nothing awaiting it, so + * nothing is in a position to catch a thrown error the way a rejected + * promise would be caught by a consumer's `for await`. A file that + * disappears between the size check and the read -- rotation schemes that + * unlink-and-recreate rather than truncate-in-place can do this -- must + * therefore be handled inside `poll()` itself; letting it throw out of a + * timer callback would crash the whole process instead of merely losing a + * line. + */ +import { closeSync, existsSync, openSync, readSync, statSync } from "node:fs"; + +export type EnvelopeLogDirection = "request" | "response"; + +/** One line of the envelope log the Guardian writes. */ +export type EnvelopeLogEntry = { + seq: number; + recorded_at: string; + direction: EnvelopeLogDirection; + method: string | null; + rpc_id: string | number | null; + envelope: unknown; +}; + +export type TailEnvelopeLogOptions = { + path: string; + /** Replay everything already in the file before following. Default false: + * start at the current end, like `tail -f`. */ + fromStart?: boolean; + pollMs?: number; + signal?: AbortSignal; + /** Called per unparseable line. Defaults to one stderr warning. Streaming + * continues either way -- a corrupt line is not a reason to stop showing + * the ones after it. */ + onMalformedLine?: (line: string, error: unknown) => void; +}; + +const NEWLINE = 0x0a; + +/** + * The envelope log is a plain file on disk; anything can write a line to it + * that is valid JSON but not a valid EnvelopeLogEntry (a number where + * recorded_at should be a string, a missing direction, ...). + * `renderEnvelopeLogEntry`'s `clockOf` calls `.slice` on `recorded_at` + * unconditionally, so an unchecked cast would let such a line reach the + * renderer and throw -- inside a `for await` loop, that kills the whole + * stream. Checked here instead, right after `JSON.parse`, using exactly the + * fields the renderer depends on. `envelope` is deliberately left + * unconstrained: it is `unknown` by design, not a shape this function's job + * to police. + */ +function isEnvelopeLogEntryShape(value: unknown): value is EnvelopeLogEntry { + if (typeof value !== "object" || value === null) { + return false; + } + const candidate = value as Record; + return ( + typeof candidate.seq === "number" && + typeof candidate.recorded_at === "string" && + (candidate.direction === "request" || candidate.direction === "response") && + (typeof candidate.method === "string" || candidate.method === null) && + (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) + ); +} + +export function tailEnvelopeLog({ + path, + fromStart = false, + pollMs = 120, + signal, + onMalformedLine = warnMalformedLine, +}: TailEnvelopeLogOptions): AsyncGenerator { + let offset = fromStart ? 0 : sizeOf(path); + // Bytes, not a string: a poll can land mid-line and, worse, mid-codepoint. + // Decoding only complete lines keeps multi-byte UTF-8 intact. + let pending = Buffer.alloc(0); + + // Entries the timer has parsed but nobody has consumed yet, and the + // wake-up the drain loop below is currently parked on while that queue is + // empty. The timer and the generator only communicate through these two. + const ready: EnvelopeLogEntry[] = []; + let wake: (() => void) | undefined; + let stopped = false; + let timer: ReturnType | undefined; + + function poll(): void { + if (stopped) { + return; + } + try { + const size = sizeOf(path); + + if (size < offset) { + // Truncated or rotated underneath us (`: > .acs/envelopes.jsonl`). + offset = 0; + pending = Buffer.alloc(0); + } + + if (size > offset) { + pending = Buffer.concat([pending, readRange(path, offset, size - offset)]); + offset = size; + + let added = false; + try { + let newline = pending.indexOf(NEWLINE); + while (newline !== -1) { + const line = pending.subarray(0, newline).toString("utf8"); + pending = pending.subarray(newline + 1); + newline = pending.indexOf(NEWLINE); + + if (line.trim() === "") { + continue; + } + try { + const parsed: unknown = JSON.parse(line); + if (!isEnvelopeLogEntryShape(parsed)) { + throw new Error("line parsed as JSON but does not match the EnvelopeLogEntry shape"); + } + ready.push(parsed); + added = true; + } catch (error) { + // `onMalformedLine` is caller-supplied and may throw. Guarded + // here rather than left to poll()'s outer catch, which would + // abandon the rest of this batch: `offset` is already at + // `size`, so any complete line still sitting in `pending` + // would never be re-scanned -- no later tick has anything new + // to read. Reporting one bad line must not cost the good ones + // behind it. + reportMalformedLine(onMalformedLine, line, error); + } + } + } finally { + // In the `finally`, not after the loop: an entry already pushed to + // `ready` must reach the consumer even if the scan above left by a + // throw, rather than sit undelivered until some unrelated write or + // the abort happens to fire `wake`. + if (added) { + wake?.(); + } + } + } + } catch (error) { + // A read can lose a race against a file that vanished between the + // size check above and the read itself. There is no promise here for + // that to reject into -- this is a timer callback, not a step inside + // the generator's own call stack -- so an uncaught throw would take + // the whole process down. Skip this tick instead: the next one's + // sizeOf() sees the gap (or the file's return) and resyncs on its + // own. + // + // The only route here is a failed read: a caller-supplied + // `onMalformedLine` is guarded at its own call site above, so it never + // reaches this catch and never abandons the rest of a batch. The + // tail-envelope-log tests cover this branch through a deterministic + // EISDIR rather than through a lost race. + warnPollError(error); + } + } + + function stop(): void { + if (stopped) { + return; + } + stopped = true; + if (timer !== undefined) { + clearInterval(timer); + } + signal?.removeEventListener("abort", stop); + wake?.(); + } + + signal?.addEventListener("abort", stop, { once: true }); + if (signal?.aborted) { + stop(); + } + + return drain(); + + async function* drain(): AsyncGenerator { + // Starting the timer here, not above, means a tail nobody ever iterates + // (and nobody ever aborts) never ticks at all. + if (!stopped) { + timer = setInterval(poll, pollMs); + } + try { + while (true) { + while (ready.length === 0 && !stopped) { + await new Promise((resolve) => { + wake = resolve; + }); + wake = undefined; + } + if (ready.length === 0) { + return; + } + const entry = ready.shift(); + if (entry !== undefined) { + yield entry; + } + } + } finally { + stop(); + } + } +} + +function sizeOf(path: string): number { + if (!existsSync(path)) { + return 0; + } + try { + return statSync(path).size; + } catch { + return 0; + } +} + +function readRange(path: string, offset: number, length: number): Buffer { + const buffer = Buffer.alloc(length); + const fd = openSync(path, "r"); + try { + const read = readSync(fd, buffer, 0, length, offset); + return buffer.subarray(0, read); + } finally { + closeSync(fd); + } +} + +/** Calls the caller's malformed-line reporter without letting it break the + * scan. A reporter that throws gets one warning of its own; the line it was + * reporting is still skipped, and the lines after it are still parsed. */ +function reportMalformedLine( + onMalformedLine: (line: string, error: unknown) => void, + line: string, + error: unknown, +): void { + try { + onMalformedLine(line, error); + } catch (callbackError) { + const message = callbackError instanceof Error ? callbackError.message : String(callbackError); + console.error(`onMalformedLine threw while reporting an unparseable envelope-log line (${message})`); + } +} + +function warnMalformedLine(line: string, error: unknown): void { + const message = error instanceof Error ? error.message : String(error); + console.error(`skipping unparseable envelope-log line (${message}): ${line.slice(0, 120)}`); +} + +function warnPollError(error: unknown): void { + const message = error instanceof Error ? error.message : String(error); + console.error(`envelope log poll failed, retrying next tick (${message})`); +} diff --git a/packages/inspector/test/render.test.ts b/packages/inspector/test/render.test.ts new file mode 100644 index 00000000..a91bad65 --- /dev/null +++ b/packages/inspector/test/render.test.ts @@ -0,0 +1,266 @@ +import { describe, expect, it } from "bun:test"; +import { + outcomeMessageOf, + renderDecisionBadge, + renderEnvelopeLogEntry, + renderOutcome, + renderRpcError, + type OutcomeMessage, + type RenderOptions, +} from "../src/render.ts"; +import type { EnvelopeLogEntry } from "../src/tail-envelope-log.ts"; + +function entry(overrides: Partial): EnvelopeLogEntry { + return { + seq: 3, + recorded_at: "2026-08-09T12:04:31.221Z", + direction: "response", + method: "steps/toolCallRequest", + rpc_id: 1, + envelope: {}, + ...overrides, + }; +} + +function response(result: Record): EnvelopeLogEntry { + return entry({ envelope: { jsonrpc: "2.0", id: 1, result } }); +} + +/** The message renderEnvelopeLogEntry would build for this envelope-log + * line, for the badge tests that assert the rendered string end to end. + * Throws rather than asserting non-null inline, so a line that stopped + * carrying an outcome fails as itself instead of as a confusing `toBe` + * diff. */ +function messageOf(line: EnvelopeLogEntry): OutcomeMessage { + const message = outcomeMessageOf(line); + if (message === null) { + throw new Error("expected this entry to carry a decision or an error"); + } + return message; +} + +function badgeFor(result: Record, options?: RenderOptions): string { + return renderOutcome(messageOf(response(result)), options); +} + +describe("outcomeMessageOf -- what the renderers are told about", () => { + it("has nothing to say about a request", () => { + expect(outcomeMessageOf(entry({ direction: "request", envelope: { jsonrpc: "2.0", id: 1 } }))).toBeNull(); + }); + + it("has nothing to say about a response with no decision -- a ServerHello", () => { + expect(outcomeMessageOf(response({ negotiated_version: "0.1.0", on_decision_failure: "proceed" }))).toBeNull(); + }); + + // The badge is handed ACS fields, not a log row to dig through, so + // everything it renders is decided here. + it("narrows the ACS fields the badge renders, and drops the rest of the envelope", () => { + expect( + outcomeMessageOf( + response({ + decision: "deny", + reason_codes: ["blocked", 7], + policy_references: [{ policy_id: "agt_stock", rule_id: "blocked" }, "not an object"], + reasoning: "ignored by the badge", + }), + ), + ).toEqual({ + kind: "decision", + decision: "deny", + reason_codes: ["blocked"], + policy_references: [{ policy_id: "agt_stock", rule_id: "blocked" }], + }); + }); + + it("reports a JSON-RPC error as an error message, with a null code when it is not a number", () => { + expect(outcomeMessageOf(entry({ envelope: { jsonrpc: "2.0", id: 1, error: { code: "nope" } } }))).toEqual({ + kind: "error", + code: null, + message: "", + }); + }); + + // The discriminant is the point: a caller can tell an outcome that IS a + // decision from one that stood in for the absence of one without + // inspecting which fields happen to be present. + it("discriminates a decision from an error, so nothing has to infer which arm it holds", () => { + expect(outcomeMessageOf(response({ decision: "allow" }))?.kind).toBe("decision"); + expect(outcomeMessageOf(entry({ envelope: { jsonrpc: "2.0", id: 1, error: { code: -32010 } } }))?.kind).toBe( + "error", + ); + }); +}); + +describe("renderDecisionBadge", () => { + // The renderer needs no envelope log at all now: anything that can build + // the message can use the badge. + it("renders a message built by hand, with no log entry anywhere in sight", () => { + expect(renderDecisionBadge({ decision: "deny", reason_codes: [], policy_references: [] })).toBe("● DENY"); + }); + + it("badges a deny, with reason_codes and policy_references", () => { + const badge = badgeFor({ + decision: "deny", + reason_codes: ["destructive_shell_command_blocked"], + policy_references: [{ policy_id: "agt_stock", rule_id: "destructive_shell_command_blocked" }], + }); + + expect(badge).toBe( + "● DENY reason_codes=[destructive_shell_command_blocked] " + + "policy_references=[agt_stock#destructive_shell_command_blocked]", + ); + }); + + it("badges a plain allow", () => { + expect(badgeFor({ decision: "allow" })).toBe("○ ALLOW"); + }); + + // The reason this badge exists: a policy that fired and let the action + // proceed arrives as an ACS `allow` with a non-empty policy_references, + // and the badge is what keeps it from being buried. + // + // This package carries no policy-runtime vocabulary, so the label says + // only what ACS itself reports -- not the policy engine's own name for + // the case -- and the last assertion holds that line. + it("distinguishes an allow that carries policy_references, without naming a disposition ACS lacks", () => { + const badge = badgeFor({ + decision: "allow", + reason_codes: ["drift_detected"], + policy_references: [{ policy_id: "agt_stock", rule_id: "drift_detected" }], + }); + + expect(badge).toBe( + "◐ ALLOW (policy fired) reason_codes=[drift_detected] policy_references=[agt_stock#drift_detected]", + ); + expect(badge).not.toBe(badgeFor({ decision: "allow" })); + expect(badge).not.toContain("warn"); + }); + + // Pins current behaviour: ACS's schemas do not require `rule_id` on a + // policy_reference, so this is a real shape, not a hypothetical one. + // Dropping to the bare policy_id here is a deliberate degradation, not a + // bug -- this test exists so a future change to it is a decision, not an + // accident. + it("renders a policy_reference with no rule_id as the bare policy_id", () => { + const badge = badgeFor({ decision: "deny", policy_references: [{ policy_id: "agt_stock" }] }); + + expect(badge).toBe("● DENY policy_references=[agt_stock]"); + }); + + it("badges modify, ask, and defer", () => { + expect(badgeFor({ decision: "modify" })).toBe("◆ MODIFY"); + expect(badgeFor({ decision: "ask" })).toBe("◆ ASK"); + expect(badgeFor({ decision: "defer" })).toBe("◆ DEFER"); + }); + + // Renders through `renderOutcome`, which is what the stream renderer + // calls: an error reaches `renderRpcError`, never the decision badge. + // `renderDecisionBadge` cannot be handed one at all -- `DecisionMessage` + // has no error arm to pass it. + it("renders a JSON-RPC error as an error, not as a decision badge", () => { + const line = renderOutcome( + messageOf(entry({ envelope: { jsonrpc: "2.0", id: 1, error: { code: -32010, message: "ACS envelope failed" } } })), + ); + + expect(line).toBe("✖ ERROR -32010 ACS envelope failed"); + }); + + // Pins the fallback the old renderer had inline: a code that is not a + // number reaches the message as null and still renders as `?`. + it("renders an error whose code is not a number", () => { + expect(renderRpcError({ code: null, message: "unreadable" })).toBe("✖ ERROR ? unreadable"); + }); + + it("emits ANSI only when colour is asked for", () => { + const plain = badgeFor({ decision: "deny" }, { color: false }); + const coloured = badgeFor({ decision: "deny" }, { color: true }); + + expect(plain).toBe("● DENY"); + expect(coloured).toContain("\u001b["); + expect(coloured).toContain("DENY"); + }); + + // With color:true, painting only the glyph and decision label would make + // a coloured badge read as one coloured half and one plain half. The + // appended segments are painted dim so the whole badge reads as one unit; + // color:false stays byte-identical to the exact-string tests above. + it("paints the appended reason_codes/policy_references segments dim when coloured", () => { + const coloured = badgeFor( + { + decision: "deny", + reason_codes: ["destructive_shell_command_blocked"], + policy_references: [{ policy_id: "agt_stock", rule_id: "destructive_shell_command_blocked" }], + }, + { color: true }, + ); + + expect(coloured).toContain("\u001b[2mreason_codes=[destructive_shell_command_blocked]\u001b[0m"); + expect(coloured).toContain("\u001b[2mpolicy_references=[agt_stock#destructive_shell_command_blocked]\u001b[0m"); + }); +}); + +describe("renderEnvelopeLogEntry", () => { + it("renders a request as a header line plus pretty JSON, with no badge", () => { + const rendered = renderEnvelopeLogEntry( + entry({ + direction: "request", + seq: 1, + envelope: { jsonrpc: "2.0", method: "steps/toolCallRequest", id: 1 }, + }), + ); + + expect(rendered.split("\n")[0]).toBe("── #1 12:04:31.221 → REQUEST steps/toolCallRequest id=1"); + expect(rendered).toContain('"jsonrpc": "2.0"'); + expect(rendered).not.toContain("●"); + }); + + it("renders a response as a header line, a badge line, then pretty JSON", () => { + const rendered = renderEnvelopeLogEntry(response({ decision: "deny", reason_codes: ["blocked"] })); + const lines = rendered.split("\n"); + + expect(lines[0]).toBe("── #3 12:04:31.221 ← RESPONSE steps/toolCallRequest id=1"); + expect(lines[1]).toBe("● DENY reason_codes=[blocked]"); + expect(lines[2]).toBe("{"); + }); + + it("labels an unpaired response -- the malformed-body case -- without an id or a method", () => { + const rendered = renderEnvelopeLogEntry( + entry({ method: null, rpc_id: null, envelope: { jsonrpc: "2.0", id: null, error: { code: -32700, message: "Parse error" } } }), + ); + + expect(rendered.split("\n")[0]).toBe("── #3 12:04:31.221 ← RESPONSE (no method) (unpaired)"); + }); + + // What this checks is that the JSON *value* round trips: nothing stripped, + // nothing reordered, only whitespace reshaped. Not that the bytes round trip + // -- the envelope log stores the value the Guardian parsed, not the bytes + // the host sent. + it("changes nothing but whitespace -- the envelope value round trips through the renderer", () => { + const envelope = { jsonrpc: "2.0", id: 1, result: { decision: "allow", nested: { deep: [1, 2] } } }; + const rendered = renderEnvelopeLogEntry(entry({ envelope })); + const jsonStart = rendered.indexOf("{"); + + expect(JSON.parse(rendered.slice(jsonStart))).toEqual(envelope); + }); + + // `isEnvelopeLogEntryShape` deliberately does not constrain `envelope`, so + // a hand-written or truncated log line reaches the renderer with the key + // missing entirely. `JSON.stringify(undefined)` returns `undefined`, + // which `join` would coerce into a blank line indistinguishable from a + // real empty body. + it("marks an entry whose envelope key is absent, instead of emitting a blank body", () => { + const withoutEnvelope = { + seq: 3, + recorded_at: "2026-08-09T12:04:31.221Z", + direction: "response", + method: "steps/toolCallRequest", + rpc_id: 1, + } as unknown as EnvelopeLogEntry; + + const lines = renderEnvelopeLogEntry(withoutEnvelope).split("\n"); + + expect(lines).toHaveLength(2); + expect(lines[0]).toBe("── #3 12:04:31.221 ← RESPONSE steps/toolCallRequest id=1"); + expect(lines[1]).toBe("(no envelope recorded)"); + }); +}); diff --git a/packages/inspector/test/tail-envelope-log.test.ts b/packages/inspector/test/tail-envelope-log.test.ts new file mode 100644 index 00000000..b4d4de64 --- /dev/null +++ b/packages/inspector/test/tail-envelope-log.test.ts @@ -0,0 +1,501 @@ +import { describe, expect, it, spyOn } from "bun:test"; +import { appendFileSync, mkdirSync, mkdtempSync, rmdirSync, truncateSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { tailEnvelopeLog, type EnvelopeLogEntry } from "../src/tail-envelope-log.ts"; + +const POLL_MS = 10; + +function entryLine(seq: number, direction: "request" | "response"): string { + const entry: EnvelopeLogEntry = { + seq, + recorded_at: "2026-08-09T12:04:31.221Z", + direction, + method: "steps/toolCallRequest", + rpc_id: seq, + envelope: { jsonrpc: "2.0", id: seq }, + }; + return `${JSON.stringify(entry)}\n`; +} + +/** Collects `count` entries or rejects after `timeoutMs`, then aborts the + * generator so the test cannot hang the suite. */ +async function collect( + iterable: AsyncGenerator, + count: number, + controller: AbortController, + timeoutMs = 3000, +): Promise { + const out: EnvelopeLogEntry[] = []; + const deadline = setTimeout(() => controller.abort(), timeoutMs); + try { + for await (const entry of iterable) { + out.push(entry); + if (out.length >= count) { + break; + } + } + } finally { + clearTimeout(deadline); + controller.abort(); + } + return out; +} + +function withTempDir(run: (dir: string, path: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-tail-")); + const path = join(dir, "envelopes.jsonl"); + return run(dir, path).finally(() => { + try { + unlinkSync(path); + } catch { + // the not-yet-created case never writes one + } + rmdirSync(dir); + }); +} + +describe("tailEnvelopeLog", () => { + it("yields entries appended after the tail starts, skipping what was already there", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, pollMs: POLL_MS, signal: controller.signal }); + // Give the generator a poll to record its starting offset before the + // append lands, which is the behaviour under test. + await Bun.sleep(POLL_MS * 3); + appendFileSync(path, entryLine(2, "response")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([2]); + }); + }); + + it("yields pre-existing entries when fromStart is set", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request") + entryLine(2, "response")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + const entries = await collect(tail, 2, controller); + expect(entries.map((e) => e.seq)).toEqual([1, 2]); + expect(entries[0]?.direction).toBe("request"); + }); + }); + + it("waits for a log file that does not exist yet", async () => { + await withTempDir(async (_dir, path) => { + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + await Bun.sleep(POLL_MS * 3); + writeFileSync(path, entryLine(1, "request")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([1]); + }); + }); + + it("reassembles a line delivered in two chunks", async () => { + await withTempDir(async (_dir, path) => { + const line = entryLine(1, "request"); + const split = Math.floor(line.length / 2); + writeFileSync(path, ""); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + appendFileSync(path, line.slice(0, split)); + await Bun.sleep(POLL_MS * 3); + appendFileSync(path, line.slice(split)); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([1]); + }); + }); + + it("restarts from zero when the log is truncated underneath it", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + const first = await collectOne(tail); + expect(first?.seq).toBe(1); + + truncateSync(path, 0); + await Bun.sleep(POLL_MS * 3); + appendFileSync(path, entryLine(9, "response")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([9]); + }); + + async function collectOne(tail: AsyncGenerator): Promise { + const { value } = await tail.next(); + return value ?? undefined; + } + }); + + it("reports a malformed line and keeps streaming", async () => { + await withTempDir(async (_dir, path) => { + const malformed: string[] = []; + writeFileSync(path, "{not json\n" + entryLine(3, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([3]); + expect(malformed).toEqual(["{not json"]); + }); + }); + + it("ends when the signal aborts", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, ""); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + controller.abort(); + + const entries: EnvelopeLogEntry[] = []; + for await (const entry of tail) { + entries.push(entry); + } + expect(entries).toEqual([]); + }); + }); + + it("keeps streaming, without crashing, when the log file is unlinked and later reappears", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + const first = await collectOne(tail); + expect(first?.seq).toBe(1); + + unlinkSync(path); + await Bun.sleep(POLL_MS * 3); + writeFileSync(path, entryLine(9, "response")); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([9]); + }); + + async function collectOne(tail: AsyncGenerator): Promise { + const { value } = await tail.next(); + return value ?? undefined; + } + }); + + it("does not start polling until the caller asks for a value", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, ""); + const malformed: string[] = []; + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + appendFileSync(path, "{not json\n"); + // Several poll intervals pass with nobody ever calling next(). If a + // timer had started at construction time rather than on first + // consumption, this malformed line would already have been reported. + await Bun.sleep(POLL_MS * 5); + expect(malformed).toEqual([]); + + appendFileSync(path, entryLine(4, "request")); + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([4]); + expect(malformed).toEqual(["{not json"]); + }); + }); + + it("reassembles a line split across two writes when a poll lands on the partial write first", async () => { + await withTempDir(async (_dir, path) => { + const line = entryLine(1, "request"); + const split = Math.floor(line.length / 2); + writeFileSync(path, ""); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + appendFileSync(path, line.slice(0, split)); + // Start consuming now, while only the first chunk is on disk, so the + // (now lazily-started) timer is actually running and gets several + // ticks against a partial line before the second chunk lands. This is + // the case the test above this one no longer exercises: there, + // consumption doesn't begin until after both chunks are already + // written, so the first poll ever run sees the complete line in one + // shot. + const collecting = collect(tail, 1, controller); + await Bun.sleep(POLL_MS * 5); + appendFileSync(path, line.slice(split)); + + const entries = await collecting; + expect(entries.map((e) => e.seq)).toEqual([1]); + }); + }); + + it("reassembles a line whose split lands mid-codepoint, without corrupting the multi-byte character", async () => { + await withTempDir(async (_dir, path) => { + const note = "🎉café"; + const entry: EnvelopeLogEntry = { + seq: 7, + recorded_at: "2026-08-09T12:04:31.221Z", + direction: "request", + method: "steps/toolCallRequest", + rpc_id: 7, + envelope: { jsonrpc: "2.0", id: 7, note }, + }; + const line = `${JSON.stringify(entry)}\n`; + const bytes = Buffer.from(line, "utf8"); + const emojiStart = bytes.indexOf(Buffer.from("🎉", "utf8")); + // Split mid-way through the emoji's 4-byte UTF-8 sequence, so neither + // chunk on its own is valid UTF-8 -- exactly what a buffer that + // decoded each poll's bytes to a string before concatenating would + // corrupt into a replacement character. + const splitAt = emojiStart + 2; + writeFileSync(path, ""); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + + appendFileSync(path, bytes.subarray(0, splitAt)); + const collecting = collect(tail, 1, controller); + await Bun.sleep(POLL_MS * 5); + appendFileSync(path, bytes.subarray(splitAt)); + + const entries = await collecting; + expect(entries).toHaveLength(1); + expect(entries[0]?.envelope).toEqual({ jsonrpc: "2.0", id: 7, note }); + }); + }); + + it("reports a line that parses as JSON but has a non-string recorded_at, and keeps streaming", async () => { + await withTempDir(async (_dir, path) => { + const malformed: string[] = []; + const badLine = JSON.stringify({ + seq: 1, + recorded_at: 12345, + direction: "request", + method: "steps/toolCallRequest", + rpc_id: 1, + envelope: { jsonrpc: "2.0", id: 1 }, + }); + writeFileSync(path, `${badLine}\n${entryLine(3, "request")}`); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([3]); + expect(malformed).toEqual([badLine]); + }); + }); + + // `if (added) wake?.()` must run even when a reporter throws mid-scan: an + // entry already pushed to `ready` must not sit undelivered until some + // later write -- or the abort -- happens to fire `wake` for an unrelated + // reason. The good line here is written BEFORE the bad one so that + // `added` is already true when the throw happens. + // A throwing onMalformedLine reaches `reportMalformedLine`'s own catch, + // which prints one stderr line of its own (`onMalformedLine threw while + // reporting...`). Spied and silenced so this deliberately-adversarial + // test does not print real stderr into a clean `bun test` run, and + // asserted on so that reporting is checked at the call site rather than + // only through the caller-side `calls` counter. + it("delivers an entry parsed before a throwing onMalformedLine, without waiting for another write", async () => { + await withTempDir(async (_dir, path) => { + const errorSpy = spyOn(console, "error").mockImplementation(() => {}); + try { + let calls = 0; + writeFileSync(path, entryLine(1, "request") + "{not json\n"); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: () => { + calls += 1; + throw new Error("reporter blew up"); + }, + }); + + // Raced against a timer rather than read through `collect`, because + // `collect`'s own deadline calls `controller.abort()` -- and `stop()` + // fires `wake?.()` on the way out, which is exactly the incidental + // rescue that made this bug look benign. No further appends and no + // abort inside the window: the only thing that can deliver entry #1 + // is the wake-up the poll itself owes the drain loop. + const first = tail.next(); + const outcome = await Promise.race([ + first.then(({ value }) => (value === undefined ? "ended" : `seq:${value.seq}`)), + Bun.sleep(POLL_MS * 20).then(() => "stranded" as const), + ]); + controller.abort(); + await first; + + expect(outcome).toBe("seq:1"); + expect(calls).toBe(1); + expect(errorSpy).toHaveBeenCalledTimes(1); + } finally { + errorSpy.mockRestore(); + } + }); + }); + + // `offset` is already at `size` by the time a line is scanned, so a batch + // abandoned mid-scan strands every complete line still in `pending` -- no + // later tick has anything new to read, and they are never re-scanned. + // Guarding the reporter at its own call site is what keeps the scan going. + // Same stray-stderr source as the test above: `reportMalformedLine`'s own + // catch prints when the caller's reporter throws. Spied, silenced, and + // asserted on for the same reason. + it("keeps parsing the lines behind a bad one when onMalformedLine throws", async () => { + await withTempDir(async (_dir, path) => { + const errorSpy = spyOn(console, "error").mockImplementation(() => {}); + try { + writeFileSync(path, "{not json\n" + entryLine(2, "response") + entryLine(3, "request")); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: () => { + throw new Error("reporter blew up"); + }, + }); + + const entries = await collect(tail, 2, controller, 1000); + expect(entries.map((e) => e.seq)).toEqual([2, 3]); + expect(errorSpy).toHaveBeenCalledTimes(1); + } finally { + errorSpy.mockRestore(); + } + }); + }); + + // `poll()`'s outer try/catch is reachable, and does not need a lost race + // to get there: pointing the tail at a directory makes `existsSync` true + // and `statSync().size` non-zero, so the read is attempted and `readSync` + // throws EISDIR every tick. Without the catch, that throw would leave a + // bare timer callback and take the process down. `poll()`'s outer catch + // warns on every failed tick (`envelope log poll + // failed, retrying next tick ...`) -- the "warns and retries" behaviour + // this test's own name claims. Spied and silenced so the several EISDIR + // ticks below do not print real stderr into a clean `bun test` run, and + // asserted on (loosely: the exact tick count is timing-dependent) so the + // warning is checked at the call site instead of only inferred from the + // process surviving. + it("survives a read that throws every tick, and resyncs once the path becomes a real file", async () => { + await withTempDir(async (_dir, path) => { + const errorSpy = spyOn(console, "error").mockImplementation(() => {}); + try { + const asDirectory = `${path}.d`; + mkdirSync(asDirectory); + // Non-zero st_size for a directory, so `size > offset` and the read is + // actually attempted rather than skipped. + writeFileSync(join(asDirectory, "child"), "x"); + + const controller = new AbortController(); + const tail = tailEnvelopeLog({ path: asDirectory, fromStart: true, pollMs: POLL_MS, signal: controller.signal }); + const collecting = collect(tail, 1, controller, 2000); + + // Several ticks against the unreadable path. The process is still + // alive on the other side of this sleep, which is the assertion. + await Bun.sleep(POLL_MS * 5); + expect(errorSpy).toHaveBeenCalled(); + expect(errorSpy.mock.calls[0]?.[0]).toContain("retrying next tick"); + + unlinkSync(join(asDirectory, "child")); + rmdirSync(asDirectory); + writeFileSync(asDirectory, entryLine(6, "response")); + + const entries = await collecting; + expect(entries.map((e) => e.seq)).toEqual([6]); + unlinkSync(asDirectory); + } finally { + errorSpy.mockRestore(); + } + }); + }); + + it("reports a line that parses as JSON but is missing direction, and keeps streaming", async () => { + await withTempDir(async (_dir, path) => { + const malformed: string[] = []; + const badLine = JSON.stringify({ + seq: 1, + recorded_at: "2026-08-09T12:04:31.221Z", + method: "steps/toolCallRequest", + rpc_id: 1, + envelope: { jsonrpc: "2.0", id: 1 }, + }); + writeFileSync(path, `${badLine}\n${entryLine(3, "request")}`); + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 1, controller); + expect(entries.map((e) => e.seq)).toEqual([3]); + expect(malformed).toEqual([badLine]); + }); + }); + + // `isEnvelopeLogEntryShape` deliberately never constrains `envelope` -- it + // is `unknown` by design -- so a line whose + // `envelope` is absent, `null`, or a bare string is still a valid + // EnvelopeLogEntry as far as this function is concerned, and reaches the + // consumer rather than being reported through `onMalformedLine`. That is + // load-bearing: the Guardian must be able to record anything that crossed + // the wire, including a malformed body it never got a real envelope for + // (see server.ts's Parse-error path, which records a response with no + // paired request at all). + it("accepts a line whose envelope is absent, null, or a bare string, without reporting it as malformed", async () => { + await withTempDir(async (_dir, path) => { + const malformed: string[] = []; + const base = { + recorded_at: "2026-08-09T12:04:31.221Z", + direction: "request" as const, + method: "steps/toolCallRequest", + rpc_id: 1, + }; + const missingEnvelope = JSON.stringify({ ...base, seq: 1 }); + const nullEnvelope = JSON.stringify({ ...base, seq: 2, envelope: null }); + const stringEnvelope = JSON.stringify({ ...base, seq: 3, envelope: "not an object" }); + writeFileSync(path, `${missingEnvelope}\n${nullEnvelope}\n${stringEnvelope}\n`); + + const controller = new AbortController(); + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 3, controller); + expect(entries.map((e) => e.seq)).toEqual([1, 2, 3]); + expect(entries.map((e) => e.envelope)).toEqual([undefined, null, "not an object"]); + expect(malformed).toEqual([]); + }); + }); +}); diff --git a/slices/v1/README.md b/slices/v1/README.md index e82f94fb..fcef1fcd 100644 --- a/slices/v1/README.md +++ b/slices/v1/README.md @@ -10,4 +10,4 @@ A single Claude Code `PreToolUse` hook (`hosts/claude-code/acs-hook.ts`) that speaks ACS over the wire to a Guardian process (`packages/guardian`), which evaluates every `steps/toolCallRequest` through AGT's unforked stock policy engine (`packages/agt-bridge`, `policy/lib`) and returns a real decision. The demo above is not staged: `docs/demos/v1-runbook.md` walks through starting the Guardian, wiring the hook into a real `claude` session, and watching a destructive shell command get denied with the policy engine's own reasoning text in the transcript — then contrasts it with a harmless command running normally. -Only `pre_tool_call` is wired; there is no session state, no envelope tap, and no second host in this slice (see the watch-for and parked-items rows in the slices doc §V1). The implementation plan this slice followed, task by task, is `docs/superpowers/plans/2026-08-09-v1-one-host-one-hook.md`. +Only `pre_tool_call` is wired; there is no session state, no envelope log, and no second host in this slice (see the watch-for and parked-items rows in the slices doc §V1). The implementation plan this slice followed, task by task, is `docs/superpowers/plans/2026-08-09-v1-one-host-one-hook.md`. diff --git a/slices/v2/README.md b/slices/v2/README.md new file mode 100644 index 00000000..8f48fd24 --- /dev/null +++ b/slices/v2/README.md @@ -0,0 +1,72 @@ +# V2: Envelope Inspector + +**Demo:** Watch the ACS request and response JSON stream live while you work. + +**Master doc:** [`docs/shaping/acs-reference-impl-slices.md`](../../docs/shaping/acs-reference-impl-slices.md) §V2 — authoritative for this slice's scope. + +**Affordances:** U20, U21, N26, N50, S6 — defined in [Detail C](../../docs/shaping/acs-reference-impl-shaping.md#detail-c-affordances). + +## What this slice delivers + +The Guardian records every ACS envelope crossing its wire into a JSONL log +(`packages/guardian/src/envelope-log-sink.ts` → `.acs/envelopes.jsonl`, S6/N26), and +`bun run inspector` (`packages/inspector`) tails that log and renders each entry live: +a header line, a decision badge for responses, then the envelope as pretty JSON +(U20/U21, N50). The demo is a third terminal beside `bun run guardian` and the agent +host. [`docs/demos/v2-runbook.md`](../../docs/demos/v2-runbook.md) walks it through with +the real captured output for a deny, an allow, a schema-invalid envelope, and an +unparseable body. + +Three properties make this worth more than a log viewer: + +- **The envelope log sink is total by construction.** N26 — `createEnvelopeLogSink`, and + the `write` method on the `EnvelopeLogSink` it returns — sits on the decision path. A + write failure disables the sink for the process lifetime, reports once, and never + propagates — an observability feature must not be able to turn a governed tool call + into an ungoverned one. `packages/guardian/test/envelope-log-sink-wiring.test.ts` asserts + exactly that end to end: *"still denies `rm -rf /` when every envelope-log write + fails"*. +- **The request is recorded before validation.** An envelope that fails the schema is + the most useful thing an ACS-first reader can see, and it is exactly what disappears + if the sink sits behind the validator. R5.1 says *every* hook firing. +- **The Inspector imports nothing from the Guardian.** It re-declares `EnvelopeLogEntry` + rather than importing it, so "inspectable on the wire" is a claim about the file rather + than about our own type graph — a third-party reader of S6 has only the file, and so + does this one. Two gates in [`test/invariants.test.ts`](../../test/invariants.test.ts) + enforce it: zero AGT vocabulary and zero host vocabulary in + `packages/inspector/src` (R5.2), and no import of `guardian` or `agt-bridge` (R5.1). + The duplication is kept honest by the round-trip contract test, which exercises both + real implementations against one file. + +## What is explicitly not in this slice + +- **A schema-invalid envelope surfaces as a JSON-RPC error, not a `deny` decision.** + The Inspector renders `✖ ERROR -32010`. `N27 denyOnInvalidEnvelope()` — the affordance + that turns Guardian-side schema and bridge failures into honoured ACS `deny` + **decisions** — is **V3**, alongside `N6 applyFailurePosture()` and + `N7 validateDecision()`. +- **U23, the posture badge**, and **N51 `tailAuditSinks()`** — the negotiated + `on_decision_failure` and the count of audited fail-open proceeds — are **V3**. There + is no considered fail-open/fail-closed posture in this tree yet. +- **U22, the session chain view** — SessionContext entries and lineage — is **V6**, + which is where session state and provenance carriage land. +- **No log rotation.** S6 grows without bound. Accepted for V2 and recorded in the + slices doc; `: > .acs/envelopes.jsonl` truncates it safely mid-run because the tail + resets on truncation. + +`.acs/envelopes.jsonl` records the JSON value the Guardian parsed, unmodified — no field +stripping, no redaction, no reordering of anything we control — so it carries raw tool +arguments. It is gitignored for that reason and is never committed. + +The precision matters, and V2 first shipped this claim too strongly. The sink is handed +`await req.json()`, so it stores a JSON *value*, not the request's bytes: the parse has +already collapsed duplicate keys, canonicalised number literals (`1.0` → `1`), and +hoisted integer-like object keys ahead of the rest — and `arguments` keys are +host-controlled, so `{"0": …, "a": …}` is a shape a real host can send. Storing raw bytes +would make the entry's `envelope` field a string rather than a JSON value, which costs +the Inspector its pretty-printing and costs the round-trip contract test its subject. The +accurate sentence is the better trade, and the whole-branch review is what caught the +inaccurate one. + +The implementation plan this slice followed, task by task, is +[`docs/superpowers/plans/2026-08-09-v2-envelope-inspector.md`](../../docs/superpowers/plans/2026-08-09-v2-envelope-inspector.md). diff --git a/test/envelope-log-sink-roundtrip.test.ts b/test/envelope-log-sink-roundtrip.test.ts new file mode 100644 index 00000000..e60d297b --- /dev/null +++ b/test/envelope-log-sink-roundtrip.test.ts @@ -0,0 +1,102 @@ +import { describe, expect, it } from "bun:test"; +import { mkdtempSync, rmdirSync, unlinkSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startGuardian } from "../packages/guardian/src/index.ts"; +import { tailEnvelopeLog, type EnvelopeLogEntry } from "../packages/inspector/src/tail-envelope-log.ts"; +import { outcomeMessageOf, renderOutcome, type OutcomeMessage } from "../packages/inspector/src/render.ts"; + +/** + * The contract test for the envelope log. The Guardian writes it; the + * Inspector declares its own EnvelopeLogEntry and reads it back. + * If either side renames a field, adds a required one, or changes a type, + * this is what fails -- nothing else would, because the two never share a + * type. + */ +function toolCallEnvelope(command: string, id: number): Record { + return { + jsonrpc: "2.0", + method: "steps/toolCallRequest", + id, + params: { + acs_version: "0.1.0", + request_id: crypto.randomUUID(), + timestamp: new Date().toISOString(), + metadata: { agent_id: "agent-1", session_id: crypto.randomUUID() }, + payload: { tool: { name: "run_shell" }, arguments: { command: { value: command } } }, + }, + }; +} + +async function take( + tail: AsyncGenerator, + count: number, + controller: AbortController, +): Promise { + const out: EnvelopeLogEntry[] = []; + const deadline = setTimeout(() => controller.abort(), 5000); + try { + for await (const entry of tail) { + out.push(entry); + if (out.length >= count) { + break; + } + } + } finally { + clearTimeout(deadline); + controller.abort(); + } + return out; +} + +describe("envelope log round trip: Guardian sink -> Inspector tail -> decision badge", () => { + it("a denied tool call arrives as a paired request/response the Inspector can render", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-roundtrip-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ + port: 0, + manifestPath: "policy/manifest.yaml", + envelopeLogPath: logPath, + }); + const controller = new AbortController(); + + try { + await fetch(guardian.url, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify(toolCallEnvelope("rm -rf /", 77)), + }); + + const tail = tailEnvelopeLog({ path: logPath, fromStart: true, pollMs: 10, signal: controller.signal }); + const [request, response] = await take(tail, 2, controller); + + // Every field the Inspector's EnvelopeLogEntry declares must actually + // be present and correctly typed on what the Guardian wrote. + expect(request?.seq).toBe(1); + expect(response?.seq).toBe(2); + expect(typeof request?.recorded_at).toBe("string"); + expect(typeof response?.recorded_at).toBe("string"); + expect(request?.direction).toBe("request"); + expect(response?.direction).toBe("response"); + expect(request?.method).toBe("steps/toolCallRequest"); + expect(response?.method).toBe("steps/toolCallRequest"); + expect(request?.rpc_id).toBe(77); + expect(response?.rpc_id).toBe(77); + + // ...and the badge reads a real AGT-backed decision off it. Two steps, + // not one: outcomeMessageOf turns the log line into what the step + // reported, and the renderer renders that message. Both halves are + // exercised here deliberately -- the round trip's claim is + // that a real Guardian's real output survives all the way to a rendered + // badge, including that it arrives as a decision rather than an error. + const message = outcomeMessageOf(response as EnvelopeLogEntry); + expect(message?.kind).toBe("decision"); + expect(renderOutcome(message as OutcomeMessage)).toContain("DENY"); + } finally { + controller.abort(); + await guardian.close(); + unlinkSync(logPath); + rmdirSync(dir); + } + }); +}); diff --git a/test/invariants.test.ts b/test/invariants.test.ts index 9a17ff13..04d5387c 100644 --- a/test/invariants.test.ts +++ b/test/invariants.test.ts @@ -22,11 +22,34 @@ function stripComments(src: string): string { return noBlockComments.replace(/(? !f.includes("/test/")) - .map((f) => ({ file: f, code: stripComments(readFileSync(`${dir}/${f}`, "utf8")) })); + const files = [...new Glob("**/*.ts").scanSync(dir)].filter((f) => !isUnderTestDir(f)); + expect({ dir, sourceFiles: files.length > 0 }).toEqual({ dir, sourceFiles: true }); + return files.map((f) => ({ file: f, code: stripComments(readFileSync(`${dir}/${f}`, "utf8")) })); } /** @@ -118,4 +141,142 @@ describe("architectural invariants", () => { "stdin", ]); }); + + /** + * An ACS-first reader must be able to trace one action end to end without + * reading AGT source. The Inspector is that reader's tool, so the claim + * is only real if the tool itself knows nothing about AGT and nothing + * about any particular host: it renders ACS envelopes as data. Both term + * lists from the two gates above apply to it at once. + */ + it("the Envelope Inspector's source contains zero AGT vocabulary and zero host vocabulary", () => { + assertNoVocabulary("packages/inspector/src", [ + "agt", + "AgentControl", + "rego", + "opa", + "intervention_point", + "verdict", + // The three AGT verdict names. Listing the word "verdict" without the + // verdicts themselves is not enough: it let a badge reading + // `ALLOW (policy fired -- ACS "warn")` pass with a green suite. ACS has + // no `warn` disposition, so that string taught a reader AGT's + // vocabulary from an ACS-first tool -- exactly the leak this gate + // exists to prevent. + // + // `allow`/`deny`/`ask`/`modify`/`defer` are deliberately NOT here -- + // they are ACS's own dispositions and the Inspector must name them. + // These three are AGT's alone. + "warn", + "escalate", + "transform", + "claude", + "opencode", + "hookSpecificOutput", + "permissionDecision", + "stdin", + ]); + }); + + /** + * Envelopes are inspectable *on the wire*. If the Inspector imported the + * Guardian's types, "inspectable" would be a claim about our own type graph + * instead: any third-party reader of the log has only the file, and so does + * this one. + */ + it("the Envelope Inspector imports nothing from the Guardian or the AGT bridge", () => { + for (const { file, code } of readSourceFiles("packages/inspector/src")) { + for (const spec of ["guardian", "agt-bridge"]) { + const found = importsSpecifier(code, spec); + expect({ file, spec, found }).toEqual({ file, spec, found: false }); + } + } + }); +}); + +/** + * True when `code` names a module specifier containing `spec` in any position + * that actually creates a dependency on it. + * + * The original gate matched `from "…"` alone (whole-branch review, finding + * 6), which is the one form nobody reaching for a forbidden import by + * accident would use. Each alternative below is a real hole it left: + * + * from "guardian" the static named/default import + * import "guardian" the bare side-effect import, no `from` + * import("guardian") dynamic, and `await import("guardian")` + * import("guardian").EnvelopeLogEntry + * type position -- erased at build, still a + * compile-time dependency on the Guardian's + * type graph, which is exactly what the gate + * above forbids + * require("guardian") CJS interop + * + * `\(?` covers the parenthesised and unparenthesised forms in one pass, and + * the `i` flag closes the last hole: module resolution is case-insensitive on + * macOS, so `from "Guardian"` resolves here and the case-sensitive gate said + * nothing about it. + */ +function importsSpecifier(code: string, spec: string): boolean { + return new RegExp(`(?:from|import|require)\\s*\\(?\\s*["'][^"']*${spec}[^"']*["']`, "i").test(code); +} + +describe("the import gate itself", () => { + /** + * A gate is only worth having if it bites. These are the exact forms the + * finding listed as blind spots, asserted directly against the matcher so + * a future simplification of the regex cannot quietly reopen one of them. + */ + it("catches every import form, in any case", () => { + const caught = [ + 'import { EnvelopeLogEntry } from "guardian";', + 'import "guardian";', + 'const g = await import("guardian");', + 'type E = import("guardian").EnvelopeLogEntry;', + 'const g = require("guardian");', + 'import { EnvelopeLogEntry } from "Guardian";', + 'export { x } from "../../guardian/src/index.ts";', + ].map((line) => ({ line, found: importsSpecifier(line, "guardian") })); + + expect(caught).toEqual(caught.map(({ line }) => ({ line, found: true }))); + }); + + it("stays quiet on code that merely mentions the word", () => { + const ignored = [ + 'const label = "guardian";', + "const guardian = startGuardian();", + 'import { renderEnvelopeLogEntry } from "./render.ts";', + ].map((line) => ({ line, found: importsSpecifier(line, "guardian") })); + + expect(ignored).toEqual(ignored.map(({ line }) => ({ line, found: false }))); + }); +}); + +describe("the source-file filter itself", () => { + /** + * `Glob.scanSync`'s relative paths never carry a leading slash, so a + * `test/` directory sitting directly under the scanned root -- rather than + * nested deeper -- produces a path with no `/test/` substring at all. No + * scanned package has such a directory today (each puts `test/` as a + * sibling of `src/`, never inside it), so this was stricter-than-intended + * rather than a real hole, but it is still the exact case a bare + * `.includes("/test/")` misses. + */ + it("excludes a test/ segment at the start of the path, not only when nested", () => { + const paths = ["test/invariants.test.ts", "src/test/helper.ts", "packages/foo/test/bar.ts"].map((path) => ({ + path, + excluded: isUnderTestDir(path), + })); + + expect(paths).toEqual(paths.map(({ path }) => ({ path, excluded: true }))); + }); + + it("does not exclude a segment that merely starts with the letters 'test'", () => { + const paths = ["latest/foo.ts", "testing/bar.ts", "src/index.ts"].map((path) => ({ + path, + excluded: isUnderTestDir(path), + })); + + expect(paths).toEqual(paths.map(({ path }) => ({ path, excluded: false }))); + }); }); diff --git a/tsconfig.json b/tsconfig.json index f7912d56..30cdd7ff 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -5,5 +5,18 @@ "rootDir": "." }, "include": ["packages/**/*.ts", "hosts/**/*.ts", "test/**/*.ts"], - "exclude": ["**/node_modules"] + "exclude": [ + "**/node_modules", + // server.test.ts's outer-net test copies packages/guardian/src here at + // test time, one directory deeper, to force a real ENOENT (see that + // test's doc comment). A killed run can leave this behind; excluded so + // it can never look like a second, drifted copy of the Guardian's + // source to `bun run typecheck`. + "packages/guardian/tmp-schemaless-scratch", + // Same reasoning: server.test.ts's two fake validate-envelope.ts test + // doubles, each copied here alongside the rest of packages/guardian/src + // at test time. Two directories, not one -- see .gitignore for why. + "packages/guardian/tmp-undefined-message-scratch", + "packages/guardian/tmp-throwing-message-accessor-scratch" + ] }