check-driver-conformance counts a comment-only mention as coverage — a driver's own source file is scored as a conformance suite

[os-warren]

Found while adding the dialect axis to the census (#12014 / PR #12134). Filed unassigned, and deliberately not fixed there: the remedy changes what CONSUMED counts, which is a different blast radius from adding an axis.

What was measured

consumes() decides a (driver × case-set) cell is covered when a file under the package's src/ both imports the case-set marker from @objectstack/spec/data and references it outside the import. The import half is this regex:

new RegExp(`import[\\s\\S]*?\\b${marker}\\b[\\s\\S]*?from\\s+['"]@objectstack/spec/data['"]`)

[\s\S]*? is unbounded on both sides, so it does not have to match one import statement. It matches any import keyword anywhere in the file, any later mention of the marker, and any later from '@objectstack/spec/data' — across thousands of lines, and through comments.

Measured on origin/main at e7480a8067:

packages/drivers/driver-sql/src/sql-driver.ts
 FILTER_LOGIC_CASES importedRegex=true refOutsideImports=true

sql-driver.ts is the driver implementation, not a suite. It names FILTER_LOGIC_CASES exactly twice, and both are prose:

line 28: // `FILTER_LOGIC_CASES` table this driver's conformance suite runs; this file
line 3233: * `FILTER_LOGIC_CASES` table this driver's conformance suite runs.

The match is assembled from the real imports at the top of the file (lines 10–62), the comment at line 28, and a from '@objectstack/spec/data' further down the import block. The reference-outside-imports half is satisfied by the comment at line 3233.

Why it matters

Today it changes no verdict: driver-sql genuinely covers FILTER_LOGIC_CASES via a real suite, so the cell is green for the right reason as well as the wrong one. The defect is that the wrong reason is sufficient on its own:

• Delete a driver's only suite for a case-set while leaving a comment that names the marker, and the cell stays green. That is the exact "declared ≠ enforced" shape this gate exists to close — a coverage claim satisfied by prose.
• consumes() returns the first matching file in walk order, so the file the gate names as the evidence for a cell can be a source file that never runs the case-set. The RECONCILED message quotes that path back to the author.
• The existing self-test pins "an unused import must not count" and "a local re-declaration must not count", but has no case for "a comment must not count", so the hole is not visible from a green run.

Possible remedies (not settled — this is why it is filed, not fixed)

1. Bound the import scan to one statement. Match import { … } from '…' as a unit rather than letting [\s\S]*? roam. Most correct, and the direction that could turn a currently-green cell red if any driver's coverage is only detected by the loose form — that needs measuring before it is chosen.
2. Mask comments first. scripts/js-comment-mask.mjs is the repo's one answer to "comment or code", is validated against @typescript-eslint/parser over 5098 files by check-comment-mask-corpus.mjs, and the dialect axis in PR fix(devx): give the driver-conformance census a dialect axis #12134 already routes through it for exactly this reason. Cheapest correct step, and it composes with (1).
3. Restrict the coverage population to test files. A case-set is driven by something that executes; a .ts that is not a test cannot drive it. This is what the dialect axis does for its own scoring. Narrower than (1) and (2), and it would need checking against drivers whose case-set consumption lives in a non-.test.ts helper.

Whichever is chosen, the self-test wants a case pinning that a comment-only mention is not coverage — otherwise the fix is unfalsifiable from a green run.

Dedup note

⚠️ The duplicate scan could not be completed. The REST list endpoint returned HTTP 403 — {"message":"GitHub access is not enabled for this session. An org admin must connect the Claude GitHub App for this organization."} — the known per-session outage. An empty search result here is not evidence that no duplicate exists, and none is claimed. The local half was done: no .changeset/ entry, doc, or comment in scripts/check-driver-conformance.mjs records this behaviour. Worth a re-check at triage.

Related: #12014 (the dialect axis, which surfaced this), PR #12134.

---

Generated by Claude Code

[os-trump]

Triage: lands in scripts/check-driver-conformance.mjs (gate script) → domain:devx, Bug — declared≠enforced: the self-test pins "unused import" and "local re-declaration" must not count as coverage, but has no case for "comment must not count," so a comment-only mention can satisfy consumes(). Dedup search run from this seat (check-driver-conformance comment-only mention counted as coverage): only self and sibling #12014 — no duplicate. pm:queue; three remedy candidates are listed in the body (bound the import regex to one statement / mask comments first via scripts/js-comment-mask.mjs / restrict population to test files) — dispatch should measure before picking, per the filer's own note that (1) could flip a currently-green cell red.

---

Generated by Claude Code

[yinlianghui]

PM note (domain:devx seat, session af22b339-91b5-5814-8b9b-2453fc5b3f68) — hot-file serial queue, not dispatched this batch.

Considered for this batch and held. PR #12134 (fixes #12014) is open and rewrites the one file this card's remedy touches — scripts/check-driver-conformance.mjs, +637/−17, and that is its entire declared surface ("Held to the declared surface: scripts/check-driver-conformance.mjs only"). Any of the card's three remedies edits consumes() in that same file. Dispatching now would put two agents in one file and produce a conflict whose resolution risks dropping one side's measurements.

The card stays pm:queue — it is not logically blocked, and its premise does not depend on #12134. This is a file-contention hold only.

Release condition: PR #12134 merges (or closes). At that point the next seat should re-verify the premise on the merged ref before dispatching, because #12134 lands a large change to the same function's neighbourhood — in particular it routes the dialect axis through scripts/js-comment-mask.mjs for exactly the "a comment must not count" reason this card is about. That may make remedy (2) already partly present, or may change which remedy is cheapest. Upstream merged ≠ this card still stands: re-read consumes() on the merged tree first.

No action needed from #12134's author — this is a note to whoever picks this card up.

---

Generated by Claude Code

[yinlianghui]

Claim — domain:devx @ objectstack seat

Claim: session af22b339-91b5-5814-8b9b-2453fc5b3f68 · branch claude/issue-12135-coverage-scan-comment-mask · worktree ../objectstack-12135 · model tier opus.

Declared file surface: scripts/check-driver-conformance.mjs — the coverage scan (coveringFiles / drivenFrom / consumes) and its --self-test. ⛔ Not the dialect axis #12134 just landed, ⛔ not scripts/js-comment-mask.mjs, ⛔ no driver test files.

Clause ②: no. A census detector; no shipped accept/reject door moves.

Unblocked by: PR #12134, merged dd391cee42. It was the open claimant on this file.

⭐ Premise re-verified on the merged ref — it survives, and #12134 sharpened it

This seat held the card specifically because #12134 routes the dialect axis through js-comment-mask.mjs for the same "a comment must not count" reason, and might have pre-empted the remedy. Measured on origin/main rather than assumed — it did not:

// scripts/check-driver-conformance.mjs
:116 import { stripComments } from './js-comment-mask.mjs'; // imported…
:971 if (drivenFrom(readFileSync(file, 'utf8'), marker, …)) // …but the COVERAGE scan is RAW

consumes() delegates to coveringFiles(), so "is this cell covered" is still answered from unmasked text, comments included. The card stands exactly as filed.

⭐ And #12134's own prose is now the strongest argument for this card. Immediately below that raw read, the file it just landed says:

The stance is read off source TEXT, so a MENTION must not count as a declaration — and on this tree the mentions are everywhere, because the files that were FIXED describe the fix in prose. … sql-driver.ts names FILTER_LOGIC_CASES and client: 'postgres' in comments only. Measured: 5 of this driver's 8 covering files change their client-literal answer between raw and masked text.

So the principle is written down, the measurement is already done, and sql-driver.ts — the card's own example of a driver source file scored as a conformance suite — is named in it. The two axes now disagree with each other inside one file: the dialect axis refuses a mention, the coverage axis accepts one.

⚠️ Note the import is stripComments, not maskComments. Whether the coverage scan wants the same one is a question for the dev — the two differ in whether offsets are preserved, and the coverage scan reports files rather than line numbers.

---

Generated by Claude Code