reflect: prove pipelines by their real output, not a substitute

corpus/CLAUDE.learned.md

@@ -14,6 +14,9 @@ Engine-only install drops this file; reflect Accepted global rules land here.
   8. **Confirm end-to-end and keep the matrix.** [IEC 61882 — HAZOP records are reusable] Run the real pipeline source-to-sink once, capture the output, commit `rca/matrix.tsv` alongside the fix. *Gate:* the end-to-end command exits 0 **and** `rca/matrix.tsv` is in the diff. The next incident starts from the existing matrix, which is how a rediscovered class stops being rediscovered.
 
   Sources for the procedure: IEC 61882 Ed. 2.0 (2016), *Hazard and operability studies (HAZOP studies) – Application guide*, https://webstore.iec.ch/en/publication/24321 — cl. 3.1.11 admits a logical part ("step in an operational sequence"), cl. 4.2 mandates the per-part guide-word sweep, and cl. 4.1 states that developing risk-treatment actions "is not a primary objective of the HAZOP examination", which is the enumerate-before-you-fix rule. Chris Jones, "Effective Troubleshooting", ch. 12 in Beyer, Jones, Petoff & Murphy (eds.), *Site Reliability Engineering: How Google Runs Production Systems*, O'Reilly, 2016, https://sre.google/sre-book/effective-troubleshooting/ — "examining each component in turn. This strategy is also well-suited for use with data processing pipelines", and "we iteratively hypothesize potential causes for the failure and try to test those hypotheses". Richard I. Cook, *How Complex Systems Fail*, 1998 rev. 2000, https://www.adaptivecapacitylabs.com/HowComplexSystemsFail.pdf — #3, "Catastrophe requires multiple failures – single point failures are not enough." Jack Jager, "Actions or Conditions: What is the Difference and Why Does it Matter?", ARMS Reliability, 2016, https://blog.armsreliability.com/blog/actions-or-conditions-what-is-the-difference-and-why-does-it-matter — "there should never be a straight line, or even a partial straight line of causes within a cause and effect chart." Ben Liblit, Mayur Naik, Alice X. Zheng, Alex Aiken & Michael I. Jordan, "Scalable Statistical Bug Isolation", PLDI 2005, DOI 10.1145/1064978.1065014, https://theory.stanford.edu/~aiken/publications/papers/pldi05.pdf — the only surveyed software method purpose-built for a program containing multiple undiagnosed bugs, with a guarantee of one predictor per distinct defect.
+- A pipeline or worker change is proven only by one uninterrupted run of the real path, at production settings, that produces the pipeline's real external output: a workflow id, a PR number, a tag, a merged commit. A unit test, a static workflow assertion, a config read-back, and a dry run are pre-checks, not proof — each one is a substitute for the real path, and the defects live exactly at the seam the substitute skips. When the real run fails, expect the next layer to fail too: keep re-running the real path until it emits the output, rather than shipping after the first green. Four times in a row in one session, each green on its pre-check and dead on the real path: a repair-plan header fix passed its unit test then died at dispatch on an agent/model mismatch; a config fix read back correctly then killed the owner at boot on a third key (`defaultExecutionHarness`) that overrides the one edited; a nightly-release fix passed four static workflow assertions then failed on dispatch with `GitHub Actions is not permitted to create or approve pull requests`; and a session-miner fix passed a dry run that filed two sessions, then filed zero for real because the submit path shelled a script that cannot reach the running owner and the default pool name was a remote-target id. Each fix was correct; none of them made the feature work.
+- Never accept a dry run as proof of a write path. Dry run proves the decision and never the effect. Same session, twice: `mergify stack push --dry-run` listed the PRs it would create while the real push was blocked by a guard, and the session miner's dry run reported `filed 2` while the real tick reported `submit failed`.
+- When a fix changes which identity, binary, or database a process uses, verify the one the *service* resolves, not the one your shell resolves. Same session: `which invoker-cli` from an ssh login returned a 0.1.0 system install under `/usr/lib/node_modules`, while the owner process's own `PATH` resolved a different 0.1.1 binary under `~/.local/npm-global/bin`; and a worker's inventory query shelled a checkout script that opened a database holding a week-old snapshot (60 workflows, newest 8 days old) instead of the live owner's (18 workflows, newest that day).
 - Absence of a field in a projection (CLI, MCP, or API output) is not proof of absent state. Say "not projected" and find the emitter before retrying, resubmitting, or switching agents. Found via `/reflect` on a 2026-09-01 session: a task sat in `needs_input` with its reason stored by the owner but omitted by every headless query; the session guessed three causes (usage limit, agent switch, dirty branch) and resubmitted twice before reading the emitter, which then took four minutes.
 - When delegating a file-finding task to a subagent and two files could plausibly hold the same answer (a duplicate, a moved file, a same-named symbol in two packages), tell the subagent to state whether each file:line claim is "read-confirmed" (it opened the actual reference/import and traced it) or "name-matched" (it assumes the file is the one in use because the name/path looked right). A subagent that reasons by name-proximity instead of tracing the real reference can hand back a confident wrong file — a judgment call about how the subagent qualifies its own confidence, not something a mechanical check can catch. Found via `/reflect` on a 2026-08-17 session: a subagent named a dead, unused duplicate file as the source of a bug; the citation was relayed as fact for three turns before a direct Read caught it.
 - A `file:line` citation, mine or a subagent's, also names the ref it was read at: working tree, `HEAD`, `origin/<base>`, or the installed bundle. A working-tree read in a checkout with untracked or modified files under the cited path is name-matched, not read-confirmed, until the same line is shown at the ref the change will actually run on. Subagent prompts that ask for read-confirmed vs name-matched must also ask for the ref. Found via `/reflect` on a 2026-09-01 session: a fix was planned against `scripts/e2e-regression-watch.mjs:330` read on a dirty feature branch after `origin/master` had already removed it (#11593); a launch-time freshness gate was misdiagnosed from an untracked copy of `task-specification-preflight.ts`; three Explore subagents reported branch-local lines as read-confirmed. Third instance after memory `verify-bug-findings-against-master` (2026-07-20).