From 774a09ad839227d1979a54b2ddecbd0a620a21b7 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Sun, 9 Aug 2026 12:27:10 -0700 Subject: [PATCH 1/2] Record the inverse trap, where a correct prediction is thrown away Every trap in this section is over-reach: narrating past an observation, believing a negative a query could not have produced, trusting a tool that exited 0. This one runs the other way, and its remedy is not more caution because caution is what produces it. Checking a claim against measurement and dropping it when the two disagree is the right instinct. What is missing is asking which of the two is more likely to be wrong before assuming it is the prediction. A number derived from first principles that contradicts the world is at least as likely to be a finding about the world. This repository nearly tripped it on 1,052. The figure was derived three ways before any deploy proved it, and the host measured 0 shared inodes across every release for weeks. The tempting reading was that the estimate was wrong; it was exact, and the world was broken, because the restore had never run. Treating the disagreement as a defect rather than as bad arithmetic is what found it. The host tripped it in the other direction on the same mechanism and recorded it there. Kept here because the shape is re-trippable by this repository, which carries several counts that are derived rather than observed. Also removes a duplicate the general form introduced. "A rule naming a target that does not exist" now appears once as a class and once as the dead-pin incident, and the class points at the incident rather than restating it, so a reader meeting the second does not have to work out whether it is a different case. Co-Authored-By: Claude Opus 5 (1M context) --- TODO.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/TODO.md b/TODO.md index 86634c0..f3d4643 100644 --- a/TODO.md +++ b/TODO.md @@ -159,9 +159,12 @@ Each of these was hit or nearly hit, and each is cheap to re-trip. - **The hub's `main` can promote while a convergence pull request is open**, so ground truth moves underneath work that was correct when it started. It happened twice in one session on 2026-08-03, and the second time added drift the branch could not have known about. Re-run the audit against the hub ref actually carried before claiming convergence, and name that ref in the change, or the claim ages into a false one. - **Before believing a negative, establish that the check could have produced a positive.** An empty result and a clean result are the same bytes, so the question is never "did it find anything" but "could it have". The instances below are the same failure in different costumes, some of which cost a false pass and one of which has not happened yet. It is stated as a demand for evidence rather than as a warning to be careful because *be careful* has no step to perform and *prove the check can fire* does. - **A query that cannot see its target.** A review-thread poll blind to suppressed findings, a reviewer filter written in the wrong API's login form, an audit loop whose `jq` path had moved, and a journal grep for `Started` from an account not in `adm` or `systemd-journal`, where the lines exist and are not shown to it. Assert the query matched before reading what it returned, which is what `jq -e` and a non-empty check are for. - - **A rule naming a target that does not exist.** `.gitattributes` pinned two paths this repository has never carried, and the comment above them claimed the case was covered, so the one file that needed the pin sat unpinned twenty lines up. [`checks/check-eol-pins.py`](./checks/check-eol-pins.py) gates that file now; the shape is not gated anywhere. + - **A rule naming a target that does not exist.** The dead-pin trap below carries this one, since the instance and its gate are written up there. It is listed here so the class is visible from the general form rather than only from the incident. - **A filter whose precondition is unstated.** The outward pass reads an absent `X-Blog-Check` as a visitor, and the field exists only because the edge is configured to log it, so a stopped capture reads as a day of real traffic. Recorded in [`OPERATIONS.md`](./OPERATIONS.md) beside the filter. - **A tool that reports success having done nothing.** `git-restore-mtime` v2022.12 printed `1,052 files to be processed`, processed none, and exited 0 for four releases. The remedy is the same in kind: assert the outcome rather than the exit status, which is what both release paths now do. +- **A correct prediction can be discarded because the world disagrees with it.** This is the inverse of the trap above and its remedy is not more caution, because caution is what produces it. Checking a claim against measurement and dropping it when the two disagree is the right instinct; what is missing is asking **which of the two is more likely to be wrong** before assuming it is the prediction. A number derived from first principles that contradicts the world is at least as likely to be a finding about the world. + - **Nearly tripped here, on 1,052.** It was derived three ways before any deploy proved it, and the host measured 0 shared inodes across every release for weeks. The tempting reading was that the estimate was wrong. It was exact, and the world was broken: the restore had never run. Treating the disagreement as a defect rather than as an error in the arithmetic is what found it. + - The host tripped it in the other direction on the same mechanism, striking a `~670 M for ten releases` figure as wrong when it had been right all along and had simply never once been true. Recorded there beside its own resolution; kept here because the shape is re-trippable by this repository, which carries several counts derived rather than observed. - **The hub authors `scripts/pr_review.py`, and hand-rolling the review loop re-discovers its bugs.** One `status` call reports rounds, head coverage, unresolved threads, suppressed findings across every round, and whether a request was ever picked up. `wait` runs the backoff in-process, so a review wait costs one turn rather than one per poll. It is read-only by design and the mutations stay explicit, so fetch and run it rather than reimplementing it. Its README documents the traps below as the reason it exists. - **A review request can sit forever without being picked up, which looks exactly like patience.** Copilot raises a `copilot_work_started` timeline event within about half a minute of accepting; a request that never draws one is not slow, it is inert, and elapsed time cannot tell them apart. The event is REST-only. Recover by clearing the request with `union: false` and an empty `botIds`, then requesting again, after reading the pending set so a human reviewer is not dropped. - **The Copilot reviewer's login differs by API, and a wrong-form filter reads as a clean review.** REST reports `copilot-pull-request-reviewer[bot]`, GraphQL omits the suffix. A filter written in the other form matches nothing, and an empty result is indistinguishable from no findings. Assert the filter matched before trusting what it returned. From 1c1fdc6dce64150cd0f1e841f173428b8a1f374e Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Sun, 9 Aug 2026 12:30:33 -0700 Subject: [PATCH 2/2] Point the cross-reference at words that exist The reference called its target the "dead-pin trap", a name appearing nowhere, so a reader had nothing to search for. It now quotes the distinctive part of the actual entry, "reads as coverage", which is present once as the trap and once here. Quoting the whole lead phrase was tried first and stutters, since it is also the wording of the reference itself. And a comma after "above", where two clauses ran together. Both found by Copilot review on #86. Co-Authored-By: Claude Opus 5 (1M context) --- TODO.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/TODO.md b/TODO.md index f3d4643..9af7b35 100644 --- a/TODO.md +++ b/TODO.md @@ -159,10 +159,10 @@ Each of these was hit or nearly hit, and each is cheap to re-trip. - **The hub's `main` can promote while a convergence pull request is open**, so ground truth moves underneath work that was correct when it started. It happened twice in one session on 2026-08-03, and the second time added drift the branch could not have known about. Re-run the audit against the hub ref actually carried before claiming convergence, and name that ref in the change, or the claim ages into a false one. - **Before believing a negative, establish that the check could have produced a positive.** An empty result and a clean result are the same bytes, so the question is never "did it find anything" but "could it have". The instances below are the same failure in different costumes, some of which cost a false pass and one of which has not happened yet. It is stated as a demand for evidence rather than as a warning to be careful because *be careful* has no step to perform and *prove the check can fire* does. - **A query that cannot see its target.** A review-thread poll blind to suppressed findings, a reviewer filter written in the wrong API's login form, an audit loop whose `jq` path had moved, and a journal grep for `Started` from an account not in `adm` or `systemd-journal`, where the lines exist and are not shown to it. Assert the query matched before reading what it returned, which is what `jq -e` and a non-empty check are for. - - **A rule naming a target that does not exist.** The dead-pin trap below carries this one, since the instance and its gate are written up there. It is listed here so the class is visible from the general form rather than only from the incident. + - **A rule naming a target that does not exist**, written up below as the trap that *reads as coverage*, where the instance and its gate are. Listed here so the class is visible from the general form rather than only from the incident. - **A filter whose precondition is unstated.** The outward pass reads an absent `X-Blog-Check` as a visitor, and the field exists only because the edge is configured to log it, so a stopped capture reads as a day of real traffic. Recorded in [`OPERATIONS.md`](./OPERATIONS.md) beside the filter. - **A tool that reports success having done nothing.** `git-restore-mtime` v2022.12 printed `1,052 files to be processed`, processed none, and exited 0 for four releases. The remedy is the same in kind: assert the outcome rather than the exit status, which is what both release paths now do. -- **A correct prediction can be discarded because the world disagrees with it.** This is the inverse of the trap above and its remedy is not more caution, because caution is what produces it. Checking a claim against measurement and dropping it when the two disagree is the right instinct; what is missing is asking **which of the two is more likely to be wrong** before assuming it is the prediction. A number derived from first principles that contradicts the world is at least as likely to be a finding about the world. +- **A correct prediction can be discarded because the world disagrees with it.** This is the inverse of the trap above, and its remedy is not more caution, because caution is what produces it. Checking a claim against measurement and dropping it when the two disagree is the right instinct; what is missing is asking **which of the two is more likely to be wrong** before assuming it is the prediction. A number derived from first principles that contradicts the world is at least as likely to be a finding about the world. - **Nearly tripped here, on 1,052.** It was derived three ways before any deploy proved it, and the host measured 0 shared inodes across every release for weeks. The tempting reading was that the estimate was wrong. It was exact, and the world was broken: the restore had never run. Treating the disagreement as a defect rather than as an error in the arithmetic is what found it. - The host tripped it in the other direction on the same mechanism, striking a `~670 M for ten releases` figure as wrong when it had been right all along and had simply never once been true. Recorded there beside its own resolution; kept here because the shape is re-trippable by this repository, which carries several counts derived rather than observed. - **The hub authors `scripts/pr_review.py`, and hand-rolling the review loop re-discovers its bugs.** One `status` call reports rounds, head coverage, unresolved threads, suppressed findings across every round, and whether a request was ever picked up. `wait` runs the backoff in-process, so a review wait costs one turn rather than one per poll. It is read-only by design and the mutations stay explicit, so fetch and run it rather than reimplementing it. Its README documents the traps below as the reason it exists.