From a completed RLCR session (7 rounds, clean exit: all acceptance criteria met and code review passed), a sanitized methodology analysis found the loop was healthy — proportional round count (~one milestone per two rounds), no circular churn, near-zero false-positive reviews, and every blocker independently reproduced by the reviewer before being reported — but with one systemic, reactive weakness.
The systemic weakness
The single most repeated review finding across the session was a test that passed while the invariant it was named for was actually violated. It recurred in different guises:
- A conformance/enforcement test that only sampled a few representative cases and missed a whole class of violating cases in the same table.
- A read-only test whose own setup performed the side-effecting action first, so the persistent artifact already existed before the assertion ran — masking that the read path created it.
- A behavior test that only statically checked the code parses, never executing it.
- An asynchronous test that used an early-arriving proxy signal as a stand-in for "the state transition completed" — both flaky and able to pass before the real work happened.
- A latent security defect that every one of the implementer's passing tests could pass without touching, because none exercised the specific leak channel.
The implementer applied a genuine "verify by live artifact, not by assertion" discipline throughout — and still shipped these gaps repeatedly. Verifying the happy path by live artifact is not the same as adversarially verifying that the invariant cannot be violated. The reviewer became the de-facto invariant-verifier every round.
Suggested methodology improvements (shift discovery left)
- Require implementers to prove each invariant test fails on deliberately-broken code (a mutation/negation check). A test never shown to fail is not evidence.
- Generate enforcement tests from the actual surface (enumerate the real route table / field set / case space) rather than hand-listing representative samples.
- Promote reviewer-identified invariant classes into whole-surface executable guards — one invariant class was re-violated in four separate new surfaces because there was no systematic guard, only a written note.
- Run the independent test-design adversarial review per-milestone, not once at the end — it was the single highest-leverage review event (it found a real production defect that all prior green suites missed).
- Add a setup-integrity gate before round zero to confirm seeded immutable artifacts are complete (no truncation, full criteria list) before freezing them — a truncated immutable artifact forced a mid-loop discipline slip.
- Budget heavyweight full-alignment audit rounds as a distinct round type with a larger execution window, so a valid audit is not reaped mid-run under the ordinary-round budget.
- Codify that final closure requires the independent gate, never a self-reported acceptance sweep — a strength observed this session worth preserving explicitly.
- Force a schedule-or-accept decision on perpetually-queued cosmetic items at loop close, so the queue resolves rather than only accumulates.
Bottom line
The methodology performed well; its one systemic weakness was reactive rather than structural — the loop repeatedly relied on the reviewer to discover that a passing test did not actually exercise its invariant. Every suggestion above aims to shift that discovery from the reviewer's catch into the implementer's own required process.
Filed from an automated RLCR methodology-analysis phase; fully sanitized (no project-specific detail).
From a completed RLCR session (7 rounds, clean exit: all acceptance criteria met and code review passed), a sanitized methodology analysis found the loop was healthy — proportional round count (~one milestone per two rounds), no circular churn, near-zero false-positive reviews, and every blocker independently reproduced by the reviewer before being reported — but with one systemic, reactive weakness.
The systemic weakness
The single most repeated review finding across the session was a test that passed while the invariant it was named for was actually violated. It recurred in different guises:
The implementer applied a genuine "verify by live artifact, not by assertion" discipline throughout — and still shipped these gaps repeatedly. Verifying the happy path by live artifact is not the same as adversarially verifying that the invariant cannot be violated. The reviewer became the de-facto invariant-verifier every round.
Suggested methodology improvements (shift discovery left)
Bottom line
The methodology performed well; its one systemic weakness was reactive rather than structural — the loop repeatedly relied on the reviewer to discover that a passing test did not actually exercise its invariant. Every suggestion above aims to shift that discovery from the reviewer's catch into the implementer's own required process.
Filed from an automated RLCR methodology-analysis phase; fully sanitized (no project-specific detail).