Skip to content

refactor(constitution)!: split resident law from retrieved articles - #6

Open
younna-ai-opencode wants to merge 4 commits into
systemfsoftware:mainfrom
younna-ai-opencode:refactor/split-resident-retrieved
Open

refactor(constitution)!: split resident law from retrieved articles#6
younna-ai-opencode wants to merge 4 commits into
systemfsoftware:mainfrom
younna-ai-opencode:refactor/split-resident-retrieved

Conversation

@younna-ai-opencode

Copy link
Copy Markdown
Contributor

Summary

The constitution now costs half as much to obey. CONSTITUTION.md was 376 lines — roughly 5,400 tokens standing in every agent's context, in every consumer repo, on every turn, whether the work touched a domain type or a README. It is now 152 lines and ~2,500 tokens. The other 21 rules moved to CONSTITUTION-ARTICLES.md and arrive when the work reaches the artifact they govern.

The split is not filing. It is drawn on which mechanism catches the violation:

Resident — CONSTITUTION.mdRetrieved — CONSTITUTION-ARTICLES.md
ContentsPreamble, Application, Article VArticles I–IV
Rules1321
Mechanically gated0 of 13 — all gate: review13 of 21 — 10 lint, 2 type-checker, 1 mutation
Deliveryalways in contexton write or edit of a source file
Cost~2.5k tokens every turn~3.6k tokens, only when it applies

A rule whose harm fires before you would know to look it up has to be resident or it does not hold. An agent about to conceal a bypass does not go looking for CONST-W3. An agent copying the file next to it does not experience that as a CONST-S2 trigger. An agent blocked by a threshold reaches for the threshold, which is exactly when CONST-E4 has to already be present. None of those thirteen is announced by an artifact, and no gate catches any of them.

Articles I–IV are the opposite. Every one fires on something the work has already put in front of the agent — a domain type, a decision function, a boundary object, a test, a module's name — and thirteen of the twenty-one are backed by a lint, type or mutation gate that fails independently of whether the prose was in the window.

Static surface across the whole harness: 506 lines → 289, before a consumer adds its own instruction file.

Design decisions

A headers-only resident index was rejected. The obvious shape — 34 rule titles resident, full text behind a pointer — fails twice. A title is not compliable, only citable, and citing a clause instead of naming the harm is the one thing CONST-G1 forbids. Worse, a plausible stub suppresses the fetch: retrieval only works when the agent notices something is missing, and CONST-P2 — The Pure Core Has One Path reads as though the rule already arrived. The resident half carries whole rules, plus a five-row table naming artifact triggers, not rule text. "Authoring a domain type" is not something an agent can comply with instead of fetching.

The partition is deliberately not gated. The tempting key — resident means gate: review — is a field the rule's own author writes, so it certifies nothing. It is also false: CONST-B1, CONST-T2 and CONST-N3 are review-gated and artifact-announced. AGENTS.md records the criterion as a review judgement, states the precedence for rules that satisfy both branches, and says why no command decides it.

The trigger condition is the law's; the mechanism is the consumer's. The articles say deliver this on write or edit of a source file, never on read — an agent that greps, or works from a plan, never fires a read trigger. Which machinery fires it (a path-scoped rule, a pre-tool gate) is a contestable tool choice, so per CONST-G2 it stays out of the constitution and lives in each consumer's AGENTS.md.

CONST-B6 landed on main mid-flight and was carried into the retrieved half between B3 and B4, upstream's own position. Its gate is type-checker — the partition working as stated, not an exception to it.

The gate, and what review found in it

PATH was hardcoded to one file. A split alone would have left the coverage comparison green over 13 of 34 rules — the script's own docstring exists to prevent exactly that reading. PATHS is now the corpus: ids unique across it, citations resolving across it, a missing file hard-failing instead of scoring.

Review then found the fix was itself half-closed, in three places. All three are fixed here; all three were found by a fixture, not by reading the code:

What review foundMeasured beforeNow
A corpus file that exists but declares no ruleresident law emptied to placeholder: truevalid: 21 rules, exit 0fails, naming the file
AGENTS.md and README.md claimed a rule dropped in a move fails coveragedropping uncited CONST-N3valid: 33 rules, exit 0; 28 of 34 rules have no inbound citation, so this was the common caseboth documents state the real boundary; --against names every vacated id
Tolerating a file absent at the older revisiona corpus file renamed in the same commit that retitles a rule → no id reassigned, exit 0still green (renames are legal) but the uncompared file is now named on the success line

The second row is the one worth dwelling on: the false claim was written in the same change that widened the gate, by the author most convinced the gate was now sound. Two independent reviewers caught it.

Reported rather than failed, in the two green rows, because failing them would fire on every legitimate deletion — which AGENTS.md's own id policy calls correct ("the number stays vacant, forever"). A gate whose false-positive rate makes it routinely waived is what CONST-E3 prices.

Breaking: consumer migration

No shim, no alias, no forwarding pointer. Each consumer:

  1. adds a second symlink — ln -s vendor/constitution/CONSTITUTION-ARTICLES.md CONSTITUTION-ARTICLES.md;
  2. keeps @CONSTITUTION.md in its harness, and does not reference the articles the same way — that would make them resident and undo the split;
  3. wires a write/edit trigger for the articles in its own AGENTS.md, or names a situational read as the fallback when its harness has no such mechanism.

Validation

pnpm testvalid: 34 rules across 6 yaml blocks in 2 files, 9 families, exit 0. pnpm test --against <pre-split rev> passes across the split itself and reports not compared, absent at 8cc2433: CONSTITUTION-ARTICLES.md — the 21 moved rules kept their ids and titles, and the file's absence at the older revision is now stated rather than assumed.

Seven known-bad fixtures, each required to fail or to report, run against the final tree:

FixtureResult
articles file absentFAIL CONSTITUTION-ARTICLES.md: missing — the corpus is both files, and half a corpus scores exactly like a whole one
resident file present but declaring no ruleFAIL CONSTITUTION.md: parses but declares no rule
CONST-P1 left in both files after a moveFAIL CONST-P1: duplicate id
CONST-T5 dropped, still cited from the resident halfFAIL dangling citation: 'CONST-T5' is cited in CONSTITUTION.md but names no rule
broken yaml fence in the articles file's third blockFAIL CONSTITUTION-ARTICLES.md block 2 — file-local index, not the corpus-wide block 5 it printed before
uncited CONST-N3 droppedgreen + 1 id(s) vacated since HEAD: CONST-N3
corpus file renamed alongside a rule retitlegreen + not compared, absent at HEAD: ARTICLES.md

Two latent defects surfaced by running the documented commands rather than reading them. import yaml was an undeclared ambient dependency, so pnpm test could not run on a clean clone — package.json now declares it and .husky/pre-commit routes through pnpm test. And pnpm test -- --against <rev>, as documented in AGENTS.md, never worked: argparse reads the lone -- as end-of-options and the rest as positionals.

Reviewer coverage. Correctness, adversarial, project-standards and maintainability ran as independently dispatched contexts. The testing lens did not complete — its agent never returned a usable result — so its coverage is missing; the unexercised-branch enumeration and the fixture battery above are the author's own, not an independent pass. No cross-model adversarial peer ran: no second-provider CLI is installed in this environment, so the adversarial lens was the in-process fallback rather than an independent model.

Declared per CONST-W3. The first commit ships the validator change together with the corpus move it judges, which CONST-E4 tells you not to do. Validator-first is uncommittable here: .husky/pre-commit runs pnpm test, and the widened validator hard-fails on a CONSTITUTION-ARTICLES.md that the next commit creates, so the ideal sequence needs --no-verify, which AGENTS.md forbids. The direction is the opposite of E4's harm — the gate went from validating 13 of 34 rules to 34, evidenced by fixtures rather than asserted — but it is a breach and it is named rather than hidden.

Residual risk, pre-existing, out of scope. The schema check tests field presence, not content: do: "" passes, and harm / check have no structural check at all. Verified present in origin/main unchanged, so this branch neither introduced nor widened it. Fixing it means a new check, which CONST-E3 says must be priced and must name the mistake it prevents — a separate change, not a rider on this one.

New concepts

Residency as a budget, not a filing decision

Standing instructions are usually organised by topic. The useful axis is cost. Every line of an always-loaded document is paid on every turn of every agent, relevant or not — and degradation from input length alone is reported at 13.9%–85% across five models by Du et al. (arXiv 2510.05381), holding even when retrieval is perfect and the irrelevant tokens are masked out. So residency is spent, not assigned.

The question to ask of each rule is not "how important is it" but "if it were not in the window, what would catch the violation?"

flowchart TD
R{"if this rule were absent,<br/>what catches the violation?"}
R -->|"nothing — the harm lands<br/>before you would look"| A[resident: pay every turn]
R -->|"a lint, type or mutation gate"| B[retrieved: pay on the trigger]
R -->|"the work itself announces it"| B
B --> C["trigger on write/edit,<br/>never on read"]
Loading

Here that produced a clean line: thirteen rules with no mechanism behind them stayed; twenty-one artifact-announced rules, thirteen of them mechanically gated, left. The trigger fires on write or edit rather than read, because a read trigger is defeated by grepping or working from a plan.

One precedence rule is needed and was initially missing: where both branches fit — an artifact exists and the violating agent is the one who would not fetch the document — resident wins. That is why the conduct and subtraction families stay resident despite being describable by their artifacts.

When not to use it. If the retrieved half has no automatic trigger — only a "read this when relevant" line — the move converts a resident rule into a discretionary one and weakens it. Retrieval also loses to residency for anything a compaction can drop mid-task. Split by mechanism only when a mechanism actually exists.

The vacuous pass

A gate can go green because it stopped looking. This is not a bug in the checking logic; the logic is fine and it runs to completion. The input set silently shrank, and a check over a subset reports the same shape as a check over the whole.

PATH = "CONSTITUTION.md" was correct for one file. Move two thirds of the rules to a second file and the same script still parses, still compares declared ids against parsed ids, still finds no discrepancy, and still exits 0 — over 13 of 34 rules. Nothing in the output distinguishes that from a healthy corpus.

Three habits close it, and this PR needed all three:

HabitWhy
A missing input is a hard failure, never a smaller passhalf a corpus scores exactly like a whole one
Presence is not contributiona file that resolves, parses and yields nothing scores like one that yields its half
What cannot be failed must be reporteda legal deletion or a new file cannot fail without firing on correct work — so name it on the success line instead of implying full coverage

And the trap: never key the check on a value its own author supplies. The tempting fix for the first row is a pinned expected rule count, which the author writes, so the check never runs on the case it exists to catch.

Written up as a durable repo learning in docs/solutions/architecture-patterns/the-vacuous-pass-gate-input-sets.md, with the fixture battery as the artifact worth keeping.

CONSTITUTION.md stood at 376 lines / ~5.4k tokens resident in every agent's
context, in every consumer repo, on every turn. Root AGENTS.md (130) plus the
constitution (376) made a 506-line static surface before a consumer added its
own instruction file.
Split on which mechanism catches the violation, never on document order:
- CONSTITUTION.md keeps the Preamble, the Application block and Article V:
13 rules, and all 13 are gate: review. Nothing announces them and no lint,
type or mutation gate catches them, so prose is the only channel they have.
Resident, ~2.5k tokens.
- CONSTITUTION-ARTICLES.md takes Articles I-IV: 21 rules that fire on an
artifact the work already put in front of the agent, 13 of them backed by a
mechanical gate (10 lint, 2 type-checker, 1 mutation). Retrieved on write or
edit of a source file.
Rejected a headers-only resident index. A title is not compliable, only
citable, which is the one thing CONST-G1 forbids; and a plausible stub
suppresses the fetch that retrieval depends on. The resident half carries
whole rules and a five-row table naming artifact triggers, not rule text.
No shim, no alias, no forwarding pointer. Consumers add a second symlink and
wire the trigger in their own AGENTS.md.
The gate moved in this commit because it had to: PATH was hardcoded to one
file, so the split alone would have left the coverage comparison green over 13
of 34 rules. PATHS is now the corpus - ids unique across it, citations
resolving across it (CONST-S4 cites CONST-T5 over the new boundary), a missing
file hard-failing instead of scoring, and --against tolerating a file that did
not exist at the old revision. Verified against three known-bad fixtures:
absent half, duplicated id, dropped rule still cited.
CONST-B6 landed on main while this was in flight and was carried into the
retrieved half between B3 and B4, upstream's own position. Its gate is
type-checker, which is the partition working as stated.
Net lines are positive, per CONST-S4: the routing table, the articles head,
and the per-file plumbing in the validator are new. Deleted in exchange -
AGENTS.md startup step 2 (an eager read of a document already resident), one
of two @CONSTITUTION.md references, README's stale "thirty-one rules", and the
documented `pnpm test -- --against <rev>`, which argparse read as positionals
and which never worked.
package.json declares the validator's pyyaml dependency and .husky/pre-commit
routes through it; pnpm test could not run on a clean clone.
Four defects the split introduced into the validator's failure paths, all
found by review rather than by a run — every one of them only shows up in
output that is already failing, which is exactly where nobody looks twice.
- an empty-fence report for each file was appended to `errors` and then
discarded: `fail()` was called with a fresh list, so the two file-specific
lines vanished on the one path where attribution matters most
- block indices were corpus-wide while the message named a file, so a YAML
error in the articles file's third block read `block 5`. Now file-local:
the same fixture reads `CONSTITUTION-ARTICLES.md block 2`
- dangling citations were sorted per file, so ordering depended on iteration
and an id cited in both files reported twice with no dedup. Now one sorted
corpus-wide pass, one line per file that cites it, in PATHS order
- coverage still said "declared in the file"
Dropped the joined `text` copy with them. `declared_ids` and the `--against`
title map now fold per file like everything beside them, which removes a
second in-memory copy of the corpus and the cross-file regex boundary it
created — `TITLE_IN_TEXT_RE` spans two adjacent lines, so a join could match
across a file seam.
Verified against known-good plus four known-bad fixtures: absent half,
duplicated id, dropped-but-cited rule, and a YAML error in the second file
to pin the block index. Skipped one reviewer nit: swapping `open().read()`
for `pathlib.Path.read_text` buys an import and no behaviour.
Review found the split's own claim was false, and the gate's hard-fail
covered only half the shape it names.
`CONSTITUTION.md` emptied to a preamble plus two `placeholder: true` blocks
printed `valid: 21 rules across 6 yaml blocks in 2 files` and exited 0. The
missing-file arm caught absence; nothing caught presence-without-rules, which
scores identically. Every path in `PATHS` must now contribute at least one
parseable rule, and the same fixture fails naming the file.
`AGENTS.md` and `README.md` claimed `pnpm test` fails "a rule dropped in a
move". Measured: dropping uncited `CONST-N3` exits 0 at `valid: 33 rules`, and
28 of 34 rules have no inbound citation, so that was the common case rather
than an edge. Both documents now state the boundary they actually have — left
in both fails on the id, present-but-empty fails, dropped-and-cited fails,
dropped-and-uncited is reported.
Reported, not failed, because failing it would fire on every legitimate
deletion, which `AGENTS.md`'s own id policy calls correct ("the number stays
vacant, forever"). `--against` now names on the success line every id vacated
since the revision, and every corpus file it could not compare because the
file did not exist there. That second clause closes a hole this branch opened:
tolerating an absent file let a corpus file renamed in the same commit that
retitles a rule pass with "no id reassigned" — the single-file version failed
loudly there. It stays green, but no longer silent.
`AGENTS.md` § Choosing the file gained the precedence the seam was missing.
Both branches fit a conduct rule — an artifact exists AND the violator is the
one who would not fetch the document — and the tie-breaker lived only in
README, which no agent loads. Resident wins: "the work announces it" means an
artifact a complying agent has in hand.
Declared per CONST-W3: commit 1 ships the validator change together with the
corpus move it judges, which CONST-E4 tells you not to do. Validator-first is
uncommittable here — `.husky/pre-commit` runs `pnpm test`, and the widened
validator hard-fails on a `CONSTITUTION-ARTICLES.md` that the next commit
creates, so the ideal sequence requires `--no-verify`, which `AGENTS.md`
forbids. The direction is the opposite of E4's harm: the gate went from
validating 13 of 34 rules to 34, evidenced by fixtures rather than asserted.
Fixtures re-run: known-good; resident file declaring no rule (fails); uncited
rule dropped (green, vacated id named); renamed file plus retitled rule (green,
uncompared file named); absent half; duplicate id; dropped-but-cited rule;
broken fence in the second file.
A gate can go green because it stopped looking. Three shapes found this
session, each by a known-bad fixture rather than by reading the code: an
input set narrowed to one file while the corpus grew to two; a path present
and parsing but declaring nothing; and a cross-revision arm whose tolerance
for a newly created path also swallowed a renamed one.
The rules that fall out: a missing input is a hard failure never a smaller
pass, presence is not contribution, and what cannot be failed must be
reported on the success line. Plus the trap: the tempting fix is a pinned
expected count, which is a value the author supplies, so the check never
runs on the case it exists to catch.
Cited by symbol and role, no repo paths or line numbers, so it does not rot
when the validator moves.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@younna-ai-opencode