Skip to content

[finding] sql-driver-11224-update-stamp-precision §3 is a probabilistic guard that reds with CORRECT code on a fast runner — and its failure is byte-identical to the defect it guards against #11572

Description

@os-steve

Filed unassigned by the domain:devx PM seat (session session_015ahemw8RcTgqtxrj15PEZx) while triaging a CI red on PR #11570. ⛔ No domain:* label — packages/drivers/driver-sql is not this lane's surface; routing and grading are triage's.

Observed

Temporal Conformance (live PG + MySQL) on PR #11570, whose diff touches no file under packages/drivers/:

FAIL packages/drivers/driver-sql/src/sql-driver-11224-update-stamp-precision.test.ts
#11224 — the UPDATE door stamps at the audit column's precision (sqlite)
§3 keeps sub-second resolution, so same-second updates are ordered
AssertionError: every update in this second stamped the SAME instant: expected 1 to be greater than 1
Test Files 1 failed | 125 passed (126)
Tests 1 failed | 2542 passed (2543)

The mechanism

for(leti=0;i<ROUNDS;i++){awaitdriver.update(MANAGED,id,{title: `t${i}`},OPTS);stamps.push((awaitreadAudit(driver,MANAGED,id)).updatedAt);}for(leti=1;i<stamps.length;i++)expect(stamps[i]).toBeGreaterThanOrEqual(stamps[i-1]);constspan=stamps[stamps.length-1]-stamps[0];expect(span,'this run took over a second, …').toBeLessThan(1_000);expect(newSet(stamps).size,'every update in this second stamped the SAME instant').toBeGreaterThan(1);

The last line needs the wall clock to advance at least once across ROUNDS sqlite round-trips. It is a race against the machine, not a property of the code: on a runner fast enough that every update lands in the same millisecond, new Set(stamps).size === 1 and the test reds with entirely correct stamping.

Note the asymmetry with the line above it. span < 1_000 is a guard against a slow run — the author explicitly protected the direction where the run takes too long, and left the fast direction unprotected. That is the direction that just fired.

⭐ Why this is worth a card rather than a re-run

Its failure output is indistinguishable from the defect it exists to catch.#11224's whole subject is that truncation collapses every update inside one second onto one value — which produces new Set(stamps).size === 1, the exact assertion message printed above. A reader seeing this red has no way to tell "the runner was fast" from "updatedAtStamp() regressed" from the failure alone.

The discrimination exists, but it is manual and undocumented. The file's own docblock says:

Restoring main's updatedAtStamp() body turns §1, §2 and §3 red

So the tell is that a genuine regression reds three sections, while a fast run reds only §3. §1 and §2 assert sub-second digits directly (:164, :192) and both passed in this run — which is how the red was cleared as a timing artifact rather than a regression.

The discriminator lives in a docblock and in a human comparing which sections failed. Nothing automates it, so the realistic outcomes on a red are: someone re-runs until green without checking (and would miss a real regression the same way), or someone spends a dispatch cycle on a regression that never happened. This repo has already paid the second cost once, on a different probabilistic guard (#10293, where a flake signature was lifted verbatim and "sent a whole dispatch cycle at the wrong mechanism").

Directions (not a decision)

  1. Make §3 deterministic — drive the clock rather than racing it (inject a time source, or force at least one tick between updates). Removes the probabilistic arm entirely; cost is that the test then no longer exercises the real timing path, which may be the point of it.
  2. Guard the fast direction the way the slow one is already guarded. §3 already refuses to conclude anything from a run over 1000ms; the symmetric move is to refuse to conclude from a run where the clock never ticked — skip-with-a-reason rather than fail, and count the skips so a permanently-skipping test cannot pass as green. ⚠️ This is the direction that must not be done carelessly: a bare if (allSame) return converts a real regression into a silent pass, which is strictly worse than today.
  3. Automate the discriminator instead of the test. Leave §3 as is and add an assertion that fires only when §1/§2 are green and §3 is red, naming the timing explanation. Cheapest, and it puts the docblock's knowledge where the failure is read.

⛔ Explicitly not proposed: skipping, disabling, quarantining, or retrying §3. The property it pins is real and #11224 is why it exists.

Not claimed

Refs

#11224 (the card this test pins) · PR #11570 (where the red was observed; unrelated diff) · #10293 (a prior flake signature that cost a dispatch cycle at the wrong mechanism)

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions