feat(conformance): assert the quality files and the armed gitleaks check

[LukasWodka]

Summary

Adds one property family to repo-inventory.yml and the guard that reads it: quality_files — the files every repo must carry at a fixed path. Two members, both measured develop-first on 2026-08-12 across all 20 active repos:

file	present	absent in
CLAUDE.md	19/20	devex-bootstrap
.cursor/BUGBOT.md	17/20	claude-skills, devex-bootstrap, rfcs

Both files are read by a tool, not by a reviewer who would notice them missing: CLAUDE.md is what an AI session is instructed to follow, .cursor/BUGBOT.md is what Bugbot reads before reviewing a diff. A repo without one does not go red — it quietly gets worse review than its peers, and nothing in the org reported that until now.

backend#1608

Why this is not a presence check

A bare "the path is in the tree" test is the easiest thing in this repo to write as inert verification. A required file must therefore be present, a regular file, and non-empty:

• a 0-byteCLAUDE.md satisfies existence and carries nothing → finding
• a symlink (mode 120000) resolves for cat and is not the guidance being in the repo → finding
• a directory or submodule at the path → finding
• an exempt cell whose file turns up → stale exemption, the same staleness rule the caller, copy, protection and ruleset families each apply

Fail-closed

The facts come out of the tree read_repo already fetches, so the family costs no extra API call and inherits the existing fail-closed properties. Every one of these is exit 2, never "the file is absent":

• 403 / rate limit on the branch list or tree
• a truncated tree — it omits paths, so absence is not knowledge
• a blob whose size the API did not report — it cannot be told from an empty file, and guessing either way would be the guard deciding a fact it does not have

There is no code path from a failed read to a finding or to an all-clear.

Exemptions written, with reasons

repo	file	reason
claude-skills	.cursor/BUGBOT.md	*bugbot_guide_missing — new shared anchor. UNREMEDIATED; the exemption is the finding, not a justification.
rfcs	.cursor/BUGBOT.md	*bugbot_guide_missing (same anchor)
devex-bootstrap	both	*devex_bootstrap_undisposed — the existing anchor, widened from protection-only to cover the same open disposition question (backend#1597)

The devex_bootstrap_undisposed text was broadened rather than duplicated: one repo, one open question, one reason. Two texts about one decision is how two reasons start disagreeing. Its protection claims are unchanged and still accurate.

Deliberately narrow. Exemptions were written only where the file is genuinely absent — 4 cells. release-train carries both files and claude-skills/rfcs carry CLAUDE.md, so blanket-exempting every non-train repo would have produced 3 stale-exemption findings and a red audit.

Two things deliberately NOT done

1. quality / gitleaks armed as a required check needs no new family. It is already asserted by protection_policy.required_checks on develop, staging and prod. Verified live: 16/16 train repos have it armed and 16/16 are asserted, zero mismatches — every repo where the inventory does not assert it is an exempt/divergent cell with a written reason. A parallel family would duplicate a live assertion and then drift from it. It is mutation-checked instead (below), so this PR proves the mechanism is not inert rather than adding a second one.

2. .gitleaks.toml is not modelled. It is 7/20 and that is correct: a per-repo allowlist you add when you hit a false positive, not a control. Modelling it would add nine exemption rows for zero security value and make a tuning file read as a security gap — the inert-verification pattern backend#1729 exists to catch.

caller-drift.yml's header now also records why quality_files is not remediable by --create-prs, and that this one is a deliberate refusal rather than a technical limit: auto-generating a placeholder would turn every finding green while adding nothing a tool can use.

Verification

Live audit, EXIT=0, all 20 repos OK in the new column (make audit, the same invocation caller-drift.yml uses):

Inventory: 20 repos x 11 reusables + 2 copies + 2 quality files + 3 branch-protection roles + 2 ruleset kinds. Audited 20 of 20 on the develop-first branch.
| repo | train | callers | copies | quality_files | protection | rulesets |
|------------------------|-------|---------|--------|---------------|------------|----------|
| `.github` | yes | OK | OK | OK | OK | OK |
| `averaging-service` | yes | OK | OK | OK | OK | OK |
| `backend` | yes | OK | OK | OK | OK | OK |
| `claude-skills` | - | OK | OK | OK | OK | OK |
| `cli` | yes | OK | OK | OK | OK | OK |
| `client` | yes | OK | OK | OK | OK | OK |
| `client-runtime` | yes | OK | OK | OK | OK | OK |
| `data-ingestors` | yes | OK | OK | OK | OK | OK |
| `design-system` | yes | OK | OK | OK | OK | OK |
| `devex-bootstrap` | - | OK | OK | OK | OK | OK |
| `docs` | yes | OK | OK | OK | OK | OK |
| `e2e-test-agent` | yes | OK | OK | OK | OK | OK |
| `frontend-app` | yes | OK | OK | OK | OK | OK |
| `model-zoo` | yes | OK | OK | OK | OK | OK |
| `release-train` | - | OK | OK | OK | OK | OK |
| `rfcs` | - | OK | OK | OK | OK | OK |
| `start-training` | yes | OK | OK | OK | OK | OK |
| `tracebloc-engine` | yes | OK | OK | OK | OK | OK |
| `tracebloc-py-package` | yes | OK | OK | OK | OK | OK |
| `tracebloc-website` | yes | OK | OK | OK | OK | OK |
No drift. Every repo read, every entry matched.

The baseline audit on develop was also EXIT=0 before this change, so the green above is the new family passing rather than an unchanged verdict.

make check: green — ruff, shellcheck (clean), house-rules (0 findings), action-pins, actionlint (0 findings), and all four selftests.

Selftest: 136 → 160 cases, 0 failures. 24 new cases: 4 on read_repo's fail-closed paths, 7 on the evaluator (both exemption directions included), 12 on the schema, 1 on the matrix column.

Not vacuous against the live fleet. Removing the three exemptions and re-evaluating the real repos produces exactly the three expected findings (rfcs ×1, devex-bootstrap ×2) — the family bites on reality, not only on fixtures.

Mutation-checked: 11 mutations, all 11 caught (applied one at a time, reverted after each; tree verified back at 160/160):

mutation	result
absent required file no longer reported	RED
a 0-byte file passes	RED
a symlink passes	RED
a directory at the path passes	RED
an exempt file that exists is ignored	RED
a blob with no size treated as empty, not unreadable	RED
a truncated tree read as a complete one	RED
the per-repo quality_files section no longer required	RED
unmatchable path shapes accepted	RED
the quality_files matrix column removed	RED
family 3: the required_checks comparison neutralised	RED

Notes for review

• No rename, and the gate job name is untouched — it is the armed required check.
• No schema-version bump: quality_files is an added key, and load_inventory rejects a repo entry that lacks the section, so an inventory without it cannot be read as "nothing required".
• Path shapes that can never match a git tree path (absolute, .., trailing slash, whitespace-padded) are rejected at load: they would assert nothing while looking like an assertion.

🤖 Generated with Claude Code

---

Note

Low Risk
Changes extend the existing read-only org audit and inventory schema; no runtime services, auth, or auto-remediation paths are altered beyond new reporting rules.

Overview
Adds a quality_files conformance family so the org audit tracks whether each repo carries CLAUDE.md and .cursor/BUGBOT.md at fixed paths—tool-facing guidance that was previously invisible when missing.

repo-inventory.yml defines the two paths for all 20 repos with required / exempt cells and documented reasons (e.g. missing Bugbot guide on claude-skills/rfcs, devex-bootstrap disposition). .gitleaks.toml is explicitly not modeled; quality / gitleaks stays covered by existing protection assertions.

scripts/caller-drift.py loads and validates the family, reads file metadata from the same develop-first git tree as callers/copies, and evaluates presence, regular file (not symlink/dir), and non-empty content. Failed or truncated tree reads fail the repo row (exit 2), not “file absent.” The conformance matrix and report gain a quality_files column.

caller-drift.yml documents that quality_files is not auto-remediated by --create-prs (placeholders would be inert verification).

Selftests add schema, read, evaluator, and matrix coverage for the new family.

^{Reviewed by Cursor Bugbot for commit d0bcd19. Bugbot is set up for automated code reviews on this repo. Configure here.}