Skip to content

IR V6: The conformance report - #44

Open
afogel wants to merge 4 commits into
ir/v5from
ir/v6
Open

afogel wants to merge 4 commits into
ir/v5from
ir/v6

Conversation

@afogel

@afogel afogel commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Sixth slice of the Normative IR (docs/shaping/normative-ir-slices.md §V6), stacked on #43.

Demo: bun run ir verify <trace.jsonl> --guardian <dir> --deployment <dir> --hmac-key <hex> emits a W3C-style conformance report over an envelope log: per-provision verdicts with the facts that witness each violation, scoped to the profiles the session negotiated, with the non-testable and exclusion rosters printed rather than omitted, and an unexercised MAY producing no verdict. Captured output is in ir/slices/v6/README.md; ir/test/fixtures/trace/generate.ts produces the clean and violating logs the tests and CI run.

What it delivers

  • The four-layer split, complete. normalizeTrace() (N41) restates the wire as facts; validateSchemas() (N43) runs Ajv over the 44 pinned schemas, envelope and hook payload alike, into schema_violation facts; computeExternalFacts() (N42) does what Datalog cannot: RFC 8785 canonicalization, §8.2's entry_hash recomputation over the Guardian's exported entries, HMAC-SHA256 over §10's signed input with signature removed. Then the evaluator, then scopeByProfile(), applyModality(), attachEvidence() (N45 to N47).
  • Eleven verdict kinds, no dropped rows: pass, fail, not-activated (R4.3), not-exercised (R4.6), permission, non-testable, inexpressible (R4.4, R3.6), exclusion (R4.8), definition, invariant, and unevaluated.
  • unevaluated is new. A predicate that reads the Guardian's records, deployment facts, or a key the verifier was not given is reported as not evaluated with the missing relation named, never as a pass on an empty relation. The compiler's per-provision external-fact list from V5 makes this mechanical.
  • Evidence (R3.4). Subject and witness columns by name, plus the facts from the predicate's relations that mention the subject.
  • Trace format. The Guardian's envelope log or a bare JSON-RPC log; the Guardian's records as a dump directory (context-entries.jsonl plus facts/); deployment facts as .facts. Nothing imports from packages/ (R8.2).
  • Two vocabulary corrections from building the normalizer: ask_resolved is guardian-state (Guardian-to-Approver traffic is not in the Observed Agent's log); chain_mismatch_observed is derived from metadata.session_state.chain_hash against the last published head.

Results

  • Clean trace: all 21 compiled provisions pass; 1 not-exercised, 1 permission, 1 non-testable, 1 exclusion, 3 definitions/invariants.
  • Violating trace: fails exactly the nine provisions it breaches (0001, 0003, 0004, 0005, 0007, 0010, 0013, 0016, 0018), with evidence.
  • The same trace with no key, dump, or deployment facts: 13 unevaluated, named.

Verification

  • bun run typecheck passes; bun test ir: 173 tests pass (Soufflé present locally).
  • compile --check, render --check, lint, differential: green. CI gains a step that generates the clean trace and requires verify to exit 0.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BX56nQPAsqYihopWfWy1xm

The sixth slice of the Normative IR (V6 in docs/shaping/normative-ir-slices.md),
stacked on V5. `acs-ir verify <trace.jsonl>` takes the Guardian's envelope log
(or a bare JSON-RPC log), normalizes the wire into the vocabulary's relations,
validates every envelope and hook payload with Ajv against the 44 pinned
schemas, computes in ordinary code what Datalog cannot (RFC 8785
canonicalization, §8.2's entry_hash recomputation over the Guardian's exported
entries, HMAC-SHA256 verification over §10's signed input), runs the evaluator,
and then applies the scoping layer: profile activation, modality, evidence.

Every provision gets one of eleven verdicts and none is a dropped row: pass,
fail, not-activated, not-exercised, permission, non-testable, inexpressible,
exclusion, definition, invariant, and unevaluated. The last is the one the plan
did not name: a predicate that reads the Guardian's own records, deployment
configuration, or a key the verifier was not handed is reported as not
evaluated with the missing relation named, never as a pass on an empty
relation. The compiler's per-provision external-fact list is what makes that
mechanical. Each violation carries its subject and witness columns by name and
the facts that mention the subject, so a failure reads without re-running the
verifier.

The report lists the non-testable and exclusion rosters, what was not
activated, and what was not evaluated. On the generated traces the clean one
passes all 21 compiled provisions and the violating one fails the nine it
breaches. Two vocabulary corrections came from building the normalizer:
ask_resolved is Guardian state, since Guardian-to-Approver traffic is not in
the Observed Agent's log, and chain_mismatch_observed is derived from the
request's metadata.session_state.chain_hash against the last published head.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX56nQPAsqYihopWfWy1xm
@github-actions

Copy link
Copy Markdown

Normative impact (ACS 0.1.2 at 6fce2a0)

No failures, nothing to review.

Changed provisions and their tests (U22)

None.

Added provisions with no conformance test (U23)

None.

Removed provisions missing a tombstone (U24)

None.

Unmarked normative statements (U25)

None.

The repository's .gitignore excludes every dist/ directory, so
ir/dist/provision-index.md (and rules.dl from V5 on) never reached git
even though `render --check` and `compile --check` require the committed
copy. Re-include ir/dist/ and commit the files the checks compare against.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX56nQPAsqYihopWfWy1xm
@afogel
afogel added this pull request to stack #48 September 14, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants