Sharpen the hot-tier lesson: a check must report whether it was in a position to observe (10 instances in one day)

[pseudoseed]

lessons-critical.md already carries the weaker form:

"I could not tell" must never be spelled the same way as "no". A truncation, an unreachable
API, and a server too old to answer each need their own signal...

That is correct and it has not been enough. On 2026-08-29 alone, across four agents and six
subsystems, the same defect appeared ten times — including twice inside instrumentation
written that same hour specifically to avoid it.

The sharper form

A builder stated it better than the current wording, while catching its own instrument:

It is the default failure of any check that reports a value without reporting whether it was in
a position to observe one.

That is the version worth having. "Could not tell vs no" describes a symptom you notice
afterwards. "Was the check in a position to observe?" is a question you can ask while writing
it, and it catches the cases the current phrasing does not — because in most of the ten below,
nobody was thinking about a negative result at all.

The ten, so the frequency is on the record

1. porch ingests the last verdict in a lane's output, and counts a lane done because its file exists #168 — porch's findVerdict scans last→first, so a lane emitting REQUEST_CHANGES then
   APPROVE is recorded as APPROVE.
2. porch truncates a failed check to its first 5 lines, hiding the failure #172 — a failing check prints its first 5 lines; vitest puts the signal at the end.
3. Starvation notice promises an automatic ESC/Ctrl+C that cannot fire for a busy agent #190 — the starvation notice promises an automatic recovery that structurally cannot fire
   for a busy agent.
4. pgrep -f / pkill -f match agent PROMPT text — record the rule in lessons-learned #193 — pgrep -f matches agent prompt text, so a guard reported "a producer is running"
   when none was.
5. Global WebSocket used on a Node 20 minimum — third independent hit today; needs a guard, not a third fix #194 — a global WebSocket that is absent on the declared minimum Node: invisible to
   anyone on a newer one.
6. A user-text hold whose recovery fails starves silently — attempted-and-failed is spelled the same as never-attempted #198 — recovery attempted-and-failed is recorded identically to never-attempted, so the
   hold starves with no alarm.
7. opencode render-gate profile no longer matches: capture current frames, then fix, then guard against drift #197 — a rows-direction viewport clip reports no-composer-marker: "my mirror is the wrong
   height" spelled as "this app has no composer".
8. PR [Air #180] Pin the live t3 server runtime #191 — a skipped test whose skip reason was never emitted, described in a summary as
   "expected".
9. session-manager.test.ts:1331 — a 15s poll deadline expiring asserts identically to the
   valve genuinely failing. It cannot say whether it reached spawn 2 or spawn 5.
10. Two instrument failures, same hour, by two different agents: a cleanup verified with a
    pgrep pattern that never matched (reported success over 5,858 surviving files), and an
    under-load measurement whose "still running" check read process existence rather than
    ongoing work — so it sampled a finished install and would have read as a clean negative.

Items 4, 9 and 10 are the argument. Two agents violated the rule hours after writing it down,
in code written to enforce it. That is not a discipline problem.

Ask

1. Sharpen the lessons-critical.md line to lead with the observability question, keeping the
   existing "could not tell ≠ no" sentence beneath it. Hot tier is capped — this replaces and
   absorbs, it does not append.
2. Add the concrete test to lessons-learned.md (cold): when writing any check, guard, or
   measurement, state what it would report if it were not in a position to observe — and make that
   a different output from a negative.
3. Mirror to codev-skeleton/ per the dual-tree rule.

Related: #193 (asks for a checked wrapper rather than a firmer rule — the same conclusion applied
to one instance).

[pseudoseed]

The sharpest refinement of this lesson yet, and it inverts part of the framing above

Found by air-197 on 2026-08-29 while establishing why a timing bound had survived unmeasured.

spec-146-phase-9-porch-engine.test.ts carries both halves of the gate:

it.skipIf(!distBuilt)('every relative import in packed dist/ resolves inside the tarball', …)

it.skipIf(distBuilt)('records why pack relative imports could not check', () => {
 expect(packSkipReason).toBe('could not check: packages/codev/dist not built');
});

The author did the right thing. The "could not tell" case is not silent, not spelled as a pass,
and not spelled as a failure — it has its own test, its own assertion, and its own correctly-worded
reason string. This is precisely what the hot-tier lesson asks for.

And in CI, the only branch that has ever run is that one. The real assertion is skipped in the
job that includes it (no packages/codev/dist) and not collected by the job that has dist (wrong
config include). So every CI run for the life of that file has executed "could not check", passed,
and shown green.

Why this changes the ask

The existing framing — and my framing in the issue body — is that these defects come from a check
that cannot distinguish its own states. This one distinguished them perfectly and it made no
difference.

In the builder's words:

"I could not tell must never be spelled the same way as no" — here it WAS spelled differently,
and still read as "fine", because a green suite absorbs an honest "could not check" just as
completely as it absorbs a pass. Distinct wording is not enough if the channel flattens everything
to green.

That is a second, independent failure mode, and it is not fixed by any amount of care at the point
of measurement:

1. The check cannot tell its states apart — what this issue was filed about. Fixed by writing
   the check better.
2. The check tells them apart and the channel discards the distinction — this. Not fixable at
   the check at all. A pass/fail channel has exactly two values, and an honest "I was not in a
   position to observe" collapses into the good one.

Added ask

The hot-tier line should cover both. A check must report whether it was in a position to observe —
and a reporting channel must be able to carry that answer. A suite that renders "ran and passed"
and "could not run" identically is a channel that destroys the distinction its tests took care to
preserve.

Concretely, the cheapest version: skip counts are already printed (50 skipped) but nobody reads
them, and a deliberate could-not-check skip is indistinguishable from an incidental one. Something
should make a deliberate could-not-check visible — a named category, a summary line, or a CI
annotation — so that "green with 3 could-not-checks" is not read as "green".

Related: #200 (where this was found), #206 (a waiting run cannot tell waiting from overtaken — the
same collapse, in a different channel).

[pseudoseed]

Three more from spec 250, and one of them is a shape the current wording misses

Adding these as evidence rather than as an eleventh hot-tier line. The architect's read is that all
three are the same lesson already at slot 4 — "'I could not tell' must never be spelled the same way
as 'no'" — and I agree for two of them. The third is the one I think argues for the rewrite this
issue is about.

1. A churn range whose two endpoints were the same commit

tools/t3-codegen/classify-churn.mjs — --fork-drift measured upstreamBase..pin.commit.

Those were the same commit when it was written, so the range was X..X and the tool reported zero
drift, exit 0 for a fork carrying real customization commits. The tool whose entire job is
answering "what have we changed?" answered "nothing", confidently.

What makes it evidence for this issue: every test passed, with the right answer for the wrong
reason. While the endpoints were equal there was no observation to be in a position to make, so
nothing could distinguish "measured, found nothing" from "measured nothing". It only became
reachable when a later ruling froze the two commits apart.

The spec had named this exact hazard in prose — "pointed at our own fork head it compares our tree
to itself and reports no churn forever" — the plan carried it forward, and three review lanes read
the code with the warning in front of them. Prose did not help, because the code matched the prose.

2. A guard hung off a layer nothing builds

apps/server/src/persistence/Migrations.ts:173 exports MigrationsLive. Nothing in the tree
constructs it — every reference is its own definition or its own docstring. The real boot path is
persistence/Layers/Sqlite.ts's setup.

A schema guard wired to MigrationsLive would never run in production, and the obvious test — build
MigrationsLive, assert the columns appear — passes. "The layer works" and "production builds the
layer" are different claims and only the first was being tested.

Same shape as slot 4: the check was never in a position to observe production's wiring, and said
nothing about that.

3. The one I think is different: a criterion that could not be run at all

Spec 250's criterion 8b: "the server is killed partway through applying the columns and the
resulting database still opens against the pre-fork server binary."

It was tested by an in-process simulation on an in-memory database. No kill, no file, no pre-fork
binary — three of the criterion's four nouns absent, and the remaining one passed green.

Here is the part that I think the current wording does not reach. When I went to write the real
version, it could not be written with the tools that existed:

• restart is stop-then-start and refuses when nothing is running.
• start wipes the data dir before starting.

Neither can open a database the run did not just create. So a criterion about opening an existing
file had no expressible form, and had had none for as long as it had existed. Fixed by adding
start --keep-data to tools/t3-server/t3-server.mjs.

Slot 4 is about a check that answers when it could not observe. This is a rung below that: there
was no check, because the harness had no verb that could host one, and nothing anywhere reported
that absence. The criterion sat in the plan reading exactly like a criterion that passes. It took
writing the test to discover the test could not be written.

If slot 4 is rewritten to cover "in a position to observe" rather than only "answered", I would want
it to reach this case too — an unrunnable criterion is the limiting case of a check that was never in
a position to observe, and it is harder to spot than a wrong answer because there is no output to
inspect. A green 8b today is only meaningful if the next reader knows it was once green for free.

File references

Instance	File
Churn range with identical endpoints	tools/t3-codegen/classify-churn.mjs
Guard on an unbuilt layer	apps/server/src/persistence/Migrations.ts:173, real path persistence/Layers/Sqlite.ts
Unprovable criterion, and the missing verb	tools/t3-server/t3-server.mjs, tools/t3-fork/criterion-8b.mjs

Fuller write-ups in codev/reviews/250-t3code-front-end-customization.md; the generalizable half of
#1 is in codev/resources/lessons-learned.md under Testing (COLD, per the displacement ruling).