Context
Three RLCR methodology findings from a real session (GPU kernel optimization, ~8 build/review rounds, clean completion, near-zero reviewer false positives). These are deliberately scoped to gaps I did not find in the existing methodology issues (the whack-a-mole / surface-enumeration / lessons-enforcement / round-consolidation themes are already well covered by #193, #196, #207, #198 — I commented on #193 separately). Fully sanitized.
Finding 1 — Re-validate evidence inherited from an aborted/interrupted prior loop
Pattern. The session continued from a previous loop that had been aborted for an unrelated tooling misconfiguration, and deliberately did not redo the earlier "completed" baseline milestone. That carried-forward baseline turned out to be the source of the first major evidence defect (an artifact that silently lacked a whole class of required rows), which then cost a dedicated repair round several rounds later — after downstream conclusions already depended on it.
Suggestion. When an RLCR session resumes from or builds on a prior/aborted loop's artifacts, require a short inherited-evidence verification step at session start (re-run or spot-check the inherited artifacts against the acceptance criteria) before treating that milestone as done. "A milestone was marked complete in a previous loop" should not imply its evidence is still valid in the new loop.
Finding 2 — Single-pass "execute-and-capture" disposition for predicted-unsafe candidates
Pattern. A higher-risk candidate was, by the round contract's own framing, expected to be unsafe. The methodology (correctly) required it to be executed and reverted rather than silently skipped — this prevented a real deferral. But it cost ~2 rounds: one round to run-and-revert, then a separate round purely to make the failure reproducible for the record (exact command + persisted artifact path), because the first pass didn't capture that.
Suggestion. Offer a pre-registered disposition for predicted-unsafe candidates: either (a) execute-to-confirm AND capture reproducible failure evidence in the same pass (one round), or (b) record an explicit, reviewer-approved rationale for not executing. The anti-pattern to remove is splitting "run it" and "make its failure reproducible" across two rounds. The insistence on execution is good; the round-splitting is the waste.
Finding 3 — Validate tooling/config invariants at loop setup, not mid-loop
Pattern. Two infrastructure issues each cost real time mid-loop:
- A model-name string passed at loop start was subtly wrong (a missing hyphen vs the actual backend deployment name). It was accepted at setup, persisted into protected loop state, and only surfaced when the first reviewer invocation failed — forcing a full loop cancel + restart.
- The working tree lacked a built artifact directory the test harness imports from, requiring a manual workaround before any measurement could run.
Suggestion. Add a setup-time preflight to the loop start: validate that the configured reviewer/model identifier actually resolves against the configured backend (fail fast with a clear message instead of persisting a bad value into immutable state), and surface obviously-missing build/runtime prerequisites before round 0. A bad model name discovered at setup is a one-line fix; discovered after round 0 it is a cancel/restart.
Why these are worth separating
The dominant cost in our session (the serial single-edge-case review rounds) is already well-described in #193/#196/#207. These three are orthogonal: they are about trusting inherited state, disposition of foreseen-negative candidates, and failing fast on setup misconfig — none of which require changing the review/convergence machinery, only adding cheap preflight/postcondition checks.
Filed from an automated RLCR methodology-analysis phase. Sanitized for public sharing.
Context
Three RLCR methodology findings from a real session (GPU kernel optimization, ~8 build/review rounds, clean completion, near-zero reviewer false positives). These are deliberately scoped to gaps I did not find in the existing methodology issues (the whack-a-mole / surface-enumeration / lessons-enforcement / round-consolidation themes are already well covered by #193, #196, #207, #198 — I commented on #193 separately). Fully sanitized.
Finding 1 — Re-validate evidence inherited from an aborted/interrupted prior loop
Pattern. The session continued from a previous loop that had been aborted for an unrelated tooling misconfiguration, and deliberately did not redo the earlier "completed" baseline milestone. That carried-forward baseline turned out to be the source of the first major evidence defect (an artifact that silently lacked a whole class of required rows), which then cost a dedicated repair round several rounds later — after downstream conclusions already depended on it.
Suggestion. When an RLCR session resumes from or builds on a prior/aborted loop's artifacts, require a short inherited-evidence verification step at session start (re-run or spot-check the inherited artifacts against the acceptance criteria) before treating that milestone as done. "A milestone was marked complete in a previous loop" should not imply its evidence is still valid in the new loop.
Finding 2 — Single-pass "execute-and-capture" disposition for predicted-unsafe candidates
Pattern. A higher-risk candidate was, by the round contract's own framing, expected to be unsafe. The methodology (correctly) required it to be executed and reverted rather than silently skipped — this prevented a real deferral. But it cost ~2 rounds: one round to run-and-revert, then a separate round purely to make the failure reproducible for the record (exact command + persisted artifact path), because the first pass didn't capture that.
Suggestion. Offer a pre-registered disposition for predicted-unsafe candidates: either (a) execute-to-confirm AND capture reproducible failure evidence in the same pass (one round), or (b) record an explicit, reviewer-approved rationale for not executing. The anti-pattern to remove is splitting "run it" and "make its failure reproducible" across two rounds. The insistence on execution is good; the round-splitting is the waste.
Finding 3 — Validate tooling/config invariants at loop setup, not mid-loop
Pattern. Two infrastructure issues each cost real time mid-loop:
Suggestion. Add a setup-time preflight to the loop start: validate that the configured reviewer/model identifier actually resolves against the configured backend (fail fast with a clear message instead of persisting a bad value into immutable state), and surface obviously-missing build/runtime prerequisites before round 0. A bad model name discovered at setup is a one-line fix; discovered after round 0 it is a cancel/restart.
Why these are worth separating
The dominant cost in our session (the serial single-edge-case review rounds) is already well-described in #193/#196/#207. These three are orthogonal: they are about trusting inherited state, disposition of foreseen-negative candidates, and failing fast on setup misconfig — none of which require changing the review/convergence machinery, only adding cheap preflight/postcondition checks.
Filed from an automated RLCR methodology-analysis phase. Sanitized for public sharing.