Uh oh!
There was an error while loading. Please reload this page.
refactor(constitution)!: split resident law from retrieved articles - #6
Open
younna-ai-opencode wants to merge 4 commits into
Open
Conversation
CONSTITUTION.md stood at 376 lines / ~5.4k tokens resident in every agent's context, in every consumer repo, on every turn. Root AGENTS.md (130) plus the constitution (376) made a 506-line static surface before a consumer added its own instruction file. Split on which mechanism catches the violation, never on document order: - CONSTITUTION.md keeps the Preamble, the Application block and Article V: 13 rules, and all 13 are gate: review. Nothing announces them and no lint, type or mutation gate catches them, so prose is the only channel they have. Resident, ~2.5k tokens. - CONSTITUTION-ARTICLES.md takes Articles I-IV: 21 rules that fire on an artifact the work already put in front of the agent, 13 of them backed by a mechanical gate (10 lint, 2 type-checker, 1 mutation). Retrieved on write or edit of a source file. Rejected a headers-only resident index. A title is not compliable, only citable, which is the one thing CONST-G1 forbids; and a plausible stub suppresses the fetch that retrieval depends on. The resident half carries whole rules and a five-row table naming artifact triggers, not rule text. No shim, no alias, no forwarding pointer. Consumers add a second symlink and wire the trigger in their own AGENTS.md. The gate moved in this commit because it had to: PATH was hardcoded to one file, so the split alone would have left the coverage comparison green over 13 of 34 rules. PATHS is now the corpus - ids unique across it, citations resolving across it (CONST-S4 cites CONST-T5 over the new boundary), a missing file hard-failing instead of scoring, and --against tolerating a file that did not exist at the old revision. Verified against three known-bad fixtures: absent half, duplicated id, dropped rule still cited. CONST-B6 landed on main while this was in flight and was carried into the retrieved half between B3 and B4, upstream's own position. Its gate is type-checker, which is the partition working as stated. Net lines are positive, per CONST-S4: the routing table, the articles head, and the per-file plumbing in the validator are new. Deleted in exchange - AGENTS.md startup step 2 (an eager read of a document already resident), one of two @CONSTITUTION.md references, README's stale "thirty-one rules", and the documented `pnpm test -- --against <rev>`, which argparse read as positionals and which never worked. package.json declares the validator's pyyaml dependency and .husky/pre-commit routes through it; pnpm test could not run on a clean clone.
Four defects the split introduced into the validator's failure paths, all found by review rather than by a run — every one of them only shows up in output that is already failing, which is exactly where nobody looks twice. - an empty-fence report for each file was appended to `errors` and then discarded: `fail()` was called with a fresh list, so the two file-specific lines vanished on the one path where attribution matters most - block indices were corpus-wide while the message named a file, so a YAML error in the articles file's third block read `block 5`. Now file-local: the same fixture reads `CONSTITUTION-ARTICLES.md block 2` - dangling citations were sorted per file, so ordering depended on iteration and an id cited in both files reported twice with no dedup. Now one sorted corpus-wide pass, one line per file that cites it, in PATHS order - coverage still said "declared in the file" Dropped the joined `text` copy with them. `declared_ids` and the `--against` title map now fold per file like everything beside them, which removes a second in-memory copy of the corpus and the cross-file regex boundary it created — `TITLE_IN_TEXT_RE` spans two adjacent lines, so a join could match across a file seam. Verified against known-good plus four known-bad fixtures: absent half, duplicated id, dropped-but-cited rule, and a YAML error in the second file to pin the block index. Skipped one reviewer nit: swapping `open().read()` for `pathlib.Path.read_text` buys an import and no behaviour.
Review found the split's own claim was false, and the gate's hard-fail
covered only half the shape it names.
`CONSTITUTION.md` emptied to a preamble plus two `placeholder: true` blocks
printed `valid: 21 rules across 6 yaml blocks in 2 files` and exited 0. The
missing-file arm caught absence; nothing caught presence-without-rules, which
scores identically. Every path in `PATHS` must now contribute at least one
parseable rule, and the same fixture fails naming the file.
`AGENTS.md` and `README.md` claimed `pnpm test` fails "a rule dropped in a
move". Measured: dropping uncited `CONST-N3` exits 0 at `valid: 33 rules`, and
28 of 34 rules have no inbound citation, so that was the common case rather
than an edge. Both documents now state the boundary they actually have — left
in both fails on the id, present-but-empty fails, dropped-and-cited fails,
dropped-and-uncited is reported.
Reported, not failed, because failing it would fire on every legitimate
deletion, which `AGENTS.md`'s own id policy calls correct ("the number stays
vacant, forever"). `--against` now names on the success line every id vacated
since the revision, and every corpus file it could not compare because the
file did not exist there. That second clause closes a hole this branch opened:
tolerating an absent file let a corpus file renamed in the same commit that
retitles a rule pass with "no id reassigned" — the single-file version failed
loudly there. It stays green, but no longer silent.
`AGENTS.md` § Choosing the file gained the precedence the seam was missing.
Both branches fit a conduct rule — an artifact exists AND the violator is the
one who would not fetch the document — and the tie-breaker lived only in
README, which no agent loads. Resident wins: "the work announces it" means an
artifact a complying agent has in hand.
Declared per CONST-W3: commit 1 ships the validator change together with the
corpus move it judges, which CONST-E4 tells you not to do. Validator-first is
uncommittable here — `.husky/pre-commit` runs `pnpm test`, and the widened
validator hard-fails on a `CONSTITUTION-ARTICLES.md` that the next commit
creates, so the ideal sequence requires `--no-verify`, which `AGENTS.md`
forbids. The direction is the opposite of E4's harm: the gate went from
validating 13 of 34 rules to 34, evidenced by fixtures rather than asserted.
Fixtures re-run: known-good; resident file declaring no rule (fails); uncited
rule dropped (green, vacated id named); renamed file plus retitled rule (green,
uncompared file named); absent half; duplicate id; dropped-but-cited rule;
broken fence in the second file.A gate can go green because it stopped looking. Three shapes found this session, each by a known-bad fixture rather than by reading the code: an input set narrowed to one file while the corpus grew to two; a path present and parsing but declaring nothing; and a cross-revision arm whose tolerance for a newly created path also swallowed a renamed one. The rules that fall out: a missing input is a hard failure never a smaller pass, presence is not contribution, and what cannot be failed must be reported on the success line. Plus the trap: the tempting fix is a pinned expected count, which is a value the author supplies, so the check never runs on the case it exists to catch. Cited by symbol and role, no repo paths or line numbers, so it does not rot when the validator moves.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The constitution now costs half as much to obey.
CONSTITUTION.mdwas 376 lines — roughly 5,400 tokens standing in every agent's context, in every consumer repo, on every turn, whether the work touched a domain type or a README. It is now 152 lines and ~2,500 tokens. The other 21 rules moved toCONSTITUTION-ARTICLES.mdand arrive when the work reaches the artifact they govern.The split is not filing. It is drawn on which mechanism catches the violation:
CONSTITUTION.mdCONSTITUTION-ARTICLES.mdgate: reviewA rule whose harm fires before you would know to look it up has to be resident or it does not hold. An agent about to conceal a bypass does not go looking for
CONST-W3. An agent copying the file next to it does not experience that as aCONST-S2trigger. An agent blocked by a threshold reaches for the threshold, which is exactly whenCONST-E4has to already be present. None of those thirteen is announced by an artifact, and no gate catches any of them.Articles I–IV are the opposite. Every one fires on something the work has already put in front of the agent — a domain type, a decision function, a boundary object, a test, a module's name — and thirteen of the twenty-one are backed by a lint, type or mutation gate that fails independently of whether the prose was in the window.
Static surface across the whole harness: 506 lines → 289, before a consumer adds its own instruction file.
Design decisions
A headers-only resident index was rejected. The obvious shape — 34 rule titles resident, full text behind a pointer — fails twice. A title is not compliable, only citable, and citing a clause instead of naming the harm is the one thing
CONST-G1forbids. Worse, a plausible stub suppresses the fetch: retrieval only works when the agent notices something is missing, andCONST-P2 — The Pure Core Has One Pathreads as though the rule already arrived. The resident half carries whole rules, plus a five-row table naming artifact triggers, not rule text. "Authoring a domain type" is not something an agent can comply with instead of fetching.The partition is deliberately not gated. The tempting key — resident means
gate: review— is a field the rule's own author writes, so it certifies nothing. It is also false:CONST-B1,CONST-T2andCONST-N3are review-gated and artifact-announced.AGENTS.mdrecords the criterion as a review judgement, states the precedence for rules that satisfy both branches, and says why no command decides it.The trigger condition is the law's; the mechanism is the consumer's. The articles say deliver this on write or edit of a source file, never on read — an agent that greps, or works from a plan, never fires a read trigger. Which machinery fires it (a path-scoped rule, a pre-tool gate) is a contestable tool choice, so per
CONST-G2it stays out of the constitution and lives in each consumer'sAGENTS.md.CONST-B6landed onmainmid-flight and was carried into the retrieved half betweenB3andB4, upstream's own position. Its gate istype-checker— the partition working as stated, not an exception to it.The gate, and what review found in it
PATHwas hardcoded to one file. A split alone would have left the coverage comparison green over 13 of 34 rules — the script's own docstring exists to prevent exactly that reading.PATHSis now the corpus: ids unique across it, citations resolving across it, a missing file hard-failing instead of scoring.Review then found the fix was itself half-closed, in three places. All three are fixed here; all three were found by a fixture, not by reading the code:
placeholder: true→valid: 21 rules, exit 0AGENTS.mdandREADME.mdclaimed a rule dropped in a move fails coverageCONST-N3→valid: 33 rules, exit 0; 28 of 34 rules have no inbound citation, so this was the common case--againstnames every vacated idno id reassigned, exit 0The second row is the one worth dwelling on: the false claim was written in the same change that widened the gate, by the author most convinced the gate was now sound. Two independent reviewers caught it.
Reported rather than failed, in the two green rows, because failing them would fire on every legitimate deletion — which
AGENTS.md's own id policy calls correct ("the number stays vacant, forever"). A gate whose false-positive rate makes it routinely waived is whatCONST-E3prices.Breaking: consumer migration
No shim, no alias, no forwarding pointer. Each consumer:
ln -s vendor/constitution/CONSTITUTION-ARTICLES.md CONSTITUTION-ARTICLES.md;@CONSTITUTION.mdin its harness, and does not reference the articles the same way — that would make them resident and undo the split;AGENTS.md, or names a situational read as the fallback when its harness has no such mechanism.Validation
pnpm test→valid: 34 rules across 6 yaml blocks in 2 files, 9 families, exit 0.pnpm test --against <pre-split rev>passes across the split itself and reportsnot compared, absent at 8cc2433: CONSTITUTION-ARTICLES.md— the 21 moved rules kept their ids and titles, and the file's absence at the older revision is now stated rather than assumed.Seven known-bad fixtures, each required to fail or to report, run against the final tree:
FAIL CONSTITUTION-ARTICLES.md: missing — the corpus is both files, and half a corpus scores exactly like a whole oneFAIL CONSTITUTION.md: parses but declares no ruleCONST-P1left in both files after a moveFAIL CONST-P1: duplicate idCONST-T5dropped, still cited from the resident halfFAIL dangling citation: 'CONST-T5' is cited in CONSTITUTION.md but names no ruleyamlfence in the articles file's third blockFAIL CONSTITUTION-ARTICLES.md block 2— file-local index, not the corpus-wideblock 5it printed beforeCONST-N3dropped1 id(s) vacated since HEAD: CONST-N3not compared, absent at HEAD: ARTICLES.mdTwo latent defects surfaced by running the documented commands rather than reading them.
import yamlwas an undeclared ambient dependency, sopnpm testcould not run on a clean clone —package.jsonnow declares it and.husky/pre-commitroutes throughpnpm test. Andpnpm test -- --against <rev>, as documented inAGENTS.md, never worked:argparsereads the lone--as end-of-options and the rest as positionals.Reviewer coverage. Correctness, adversarial, project-standards and maintainability ran as independently dispatched contexts. The testing lens did not complete — its agent never returned a usable result — so its coverage is missing; the unexercised-branch enumeration and the fixture battery above are the author's own, not an independent pass. No cross-model adversarial peer ran: no second-provider CLI is installed in this environment, so the adversarial lens was the in-process fallback rather than an independent model.
Declared per
CONST-W3. The first commit ships the validator change together with the corpus move it judges, whichCONST-E4tells you not to do. Validator-first is uncommittable here:.husky/pre-commitrunspnpm test, and the widened validator hard-fails on aCONSTITUTION-ARTICLES.mdthat the next commit creates, so the ideal sequence needs--no-verify, whichAGENTS.mdforbids. The direction is the opposite of E4's harm — the gate went from validating 13 of 34 rules to 34, evidenced by fixtures rather than asserted — but it is a breach and it is named rather than hidden.Residual risk, pre-existing, out of scope. The schema check tests field presence, not content:
do: ""passes, andharm/checkhave no structural check at all. Verified present inorigin/mainunchanged, so this branch neither introduced nor widened it. Fixing it means a new check, whichCONST-E3says must be priced and must name the mistake it prevents — a separate change, not a rider on this one.New concepts
Residency as a budget, not a filing decision
Standing instructions are usually organised by topic. The useful axis is cost. Every line of an always-loaded document is paid on every turn of every agent, relevant or not — and degradation from input length alone is reported at 13.9%–85% across five models by Du et al. (arXiv 2510.05381), holding even when retrieval is perfect and the irrelevant tokens are masked out. So residency is spent, not assigned.
The question to ask of each rule is not "how important is it" but "if it were not in the window, what would catch the violation?"
flowchart TD R{"if this rule were absent,<br/>what catches the violation?"} R -->|"nothing — the harm lands<br/>before you would look"| A[resident: pay every turn] R -->|"a lint, type or mutation gate"| B[retrieved: pay on the trigger] R -->|"the work itself announces it"| B B --> C["trigger on write/edit,<br/>never on read"]Here that produced a clean line: thirteen rules with no mechanism behind them stayed; twenty-one artifact-announced rules, thirteen of them mechanically gated, left. The trigger fires on write or edit rather than read, because a read trigger is defeated by grepping or working from a plan.
One precedence rule is needed and was initially missing: where both branches fit — an artifact exists and the violating agent is the one who would not fetch the document — resident wins. That is why the conduct and subtraction families stay resident despite being describable by their artifacts.
When not to use it. If the retrieved half has no automatic trigger — only a "read this when relevant" line — the move converts a resident rule into a discretionary one and weakens it. Retrieval also loses to residency for anything a compaction can drop mid-task. Split by mechanism only when a mechanism actually exists.
The vacuous pass
A gate can go green because it stopped looking. This is not a bug in the checking logic; the logic is fine and it runs to completion. The input set silently shrank, and a check over a subset reports the same shape as a check over the whole.
PATH = "CONSTITUTION.md"was correct for one file. Move two thirds of the rules to a second file and the same script still parses, still compares declared ids against parsed ids, still finds no discrepancy, and still exits 0 — over 13 of 34 rules. Nothing in the output distinguishes that from a healthy corpus.Three habits close it, and this PR needed all three:
half a corpus scores exactly like a whole oneAnd the trap: never key the check on a value its own author supplies. The tempting fix for the first row is a pinned expected rule count, which the author writes, so the check never runs on the case it exists to catch.
Written up as a durable repo learning in
docs/solutions/architecture-patterns/the-vacuous-pass-gate-input-sets.md, with the fixture battery as the artifact worth keeping.