Uh oh!
There was an error while loading. Please reload this page.
fix(devx): give the driver-conformance census a dialect axis - #12134
Merged
os-warren merged 4 commits intoAug 25, 2026
Conversation
The census scored `driver x case-set` and reported 45 covered cells before and
after a change whose entire content was "this conformance suite went from
executing on one dialect to executing on three". ADR-0053 D-A3 declares the
matrix with a third axis -- `driver {SQLite, Postgres at minimum}` -- enforced
only from inside a suite, by opt-in routing through live-dialect-matrix.testkit
and by OS_EXPECT_LIVE_DIALECT_MATRIX=1. Both fire only for a file that already
opted in, so a suite hard-coding `client: 'better-sqlite3'` was invisible to
every gate in the repo while counting as a covered cell.
Measured cost, not hypothetical: sql-driver-aggregation-conformance.test.ts was
green on main and on every PR; on live PG 16.13 sum/avg/min/max over a boolean
column threw SQLSTATE 42883.
DIALECTED is the new invariant. It asserts that a conformance suite SAYS which
dialects it runs on -- not that the answer is good enough, which is a different
decision. The population is defined by the presence of a DECLARATION, never by
the absence of a client literal, so moving the config into a helper cannot
respell a suite out of the gate's reach.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6oThe UNDECLARED glyph is four characters wider than the others, so the run where the table is most worth reading was the one where its columns broke. Found by the ablation, which is the only run that produces that glyph. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o
The first cut of this axis carried its own stripComments. That is the exact anti-pattern scripts/js-comment-mask.mjs was written to delete: every source-scanning gate used to answer "comment or code" privately, and the copies drifted into two silent families -- a naive regex that opens a phantom comment on a `/*` inside a string, and a string-aware scanner that opens a phantom string on a quote inside a regex character class. The shared module is validated against @typescript-eslint/parser over the whole tree by check-comment-mask-corpus.mjs in CI; a private copy is validated by whatever its author thought of. stripComments is the documented projection for this caller: it feeds a scanner and reports neither a line number nor a byte offset. The census output is byte-identical across the swap, so this is a change of provenance and not of measurement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W6HFzyH98W1YaQXhJUJt6o
This was referenced Aug 25, 2026
os-warren
marked this pull request as ready for review
August 25, 2026 14:47
Uh oh!
There was an error while loading. Please reload this page.
os-warren
deleted the
claude/issue-12014-conformance-census-dialect-axis
branch
August 25, 2026 15:08
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes#12014
The census scored
driver × case-setand reported the same thing before and after a change whose entire content was "this conformance suite went from executing on one dialect to executing on three". ADR-0053 D-A3 declares the matrix with a third axis —driver {SQLite, Postgres at minimum}— and that axis was enforced only from inside a suite, by routing throughlive-dialect-matrix.testkit.tsand byOS_EXPECT_LIVE_DIALECT_MATRIX=1. Both are opt-in: they fire only for a file that already opted in. So a suite that hard-codedclient: 'better-sqlite3'was invisible to every gate in the repo, counted here as a covered cell, with nothing anywhere saying it measured one dialect of three.This adds the third axis to the census.
DIALECTEDasserts that a conformance suite says which dialects it runs on.What moved
The covered-cell verdict is unchanged, and that is the point — no cell's
CONSUMEDanswer moves, because nothing about coverage was wrong. What was missing is the second line:and the per-suite table above it:
The classification counts
Re-measured on
origin/mainate7480a8067, and they match the numbers the card was claimed with:driver-sql/srcwith a literalclient: 'better-sqlite3'live-dialect-matrixdialectCell(⛔ The 105 are not 105 defects and this gate does not treat them as any. Most are legitimately SQLite-specific —
sql-driver-11321-sqlite-audit-default-canonical.test.tsis about SQLite by name. The gap is that "deliberately single-dialect" and "accidentally single-dialect" were spelled identically.The gate therefore scores a much smaller population: the suites that make a cell of this census covered — the ones whose coverage this script's own headline is asserting. That is 7 files, not 105:
sql-driver.ts) is not a test file, so nothing executes and it carries no stance. It is named in the output rather than filtered away in silence.Per-cell: 8 of 9 dialect-scored cells have a matrix-routed suite. The one that does not is
FILTER_COMPARAND_TYPE_CASES, whose ownCASE_SETSentry describes it as "the six accepted types compile everywhere" while it is measured on one dialect.Why the two are ledgered rather than marked
Marking them
dialectCell('sqlite')would be a mechanical, behaviour-preserving edit — the literal in both is byte-for-byte the sqlite cell'sconfig(). It is deliberately not done here, because the marker would be recording a claim about intent that cannot be verified from the outside:sql-driver-comparand-type-conformance.test.ts— its case-set says the accepted types "compile everywhere", and this is the only dialect-scored cell with no matrix-routed suite at all. Writingdeliberately sqlitethere would write down the opposite of its own case-set's claim.sql-driver-icontains-and-retired-operators.test.ts—FILTER_TEXT_CASESis the case-set whose answer is known to diverge by dialect (drivers(sql family): 文本算子的大小写折叠是「方言的」而非「契约的」——$contains在 SQLite 过折叠、$icontains在 PG/MySQL 过折叠 #6518 made case sensitivity per-dialect:GLOBon SQLite,LIKEon Postgres,LIKEoverCAST(… AS BINARY)on MySQL). A SQLite-only suite over it is the shape most likely to be accidental, so "deliberate" is exactly the word that must not be guessed.Both are a follow-up whose size this card's number sets: 2 suites, and the question for each is a behaviour decision (convert to the matrix, or declare the single cell), not a marking. No test file is edited by this PR.
Design decisions worth reviewing
Declared, not detected. The population is defined by the presence of a declaration, never by the absence of a
client:literal. A detector keyed on the literal sees only the spellings it knows — move the config into a helper or behind abeforeAlland the literal is gone while the coverage is exactly as narrow. A suite passes by naming a stance symbol from its driver's testkit, so the gate cannot be respelled around.Per file, not per cell. A cell can be covered by two suites. Scoring the cell would let an undeclared suite hide behind a matrix-routed sibling — which is the arrangement
FILTER_TEXT_CASESis in on this tree right now.Both axes still come off disk. Dialect-capable drivers are discovered by their
DIALECT_CELLSexport, and the cell ids are read from the testkit's array body — never a hardcoded package name or dialect list.driver-memory,driver-mongodb,driver-sqlite-wasmanddriver-tursohave no such testkit, so they are single-backend and are reported as a count rather than skipped in silence.The comment mask is shared, not private. The stance is read from source text, so a mention must not count as a declaration — and on this tree the mentions are everywhere, because the files that were fixed describe the fix in prose.
sql-driver-aggregation-conformance.test.ts(the #11456 conversion) quotes the import it now uses inside its head note;sql-driver.tsnamesFILTER_LOGIC_CASESandclient: 'postgres'in comments only. Measured: 5 of this driver's 8 covering files change their client-literal answer between raw and masked text. The first cut of this branch carried its ownstripComments; that is the exact anti-patternscripts/js-comment-mask.mjsexists to delete, so it now routes through the shared module. The census output is byte-identical across that swap — a change of provenance, not of measurement.The second ledger carries the same authority as the first. A second ledger is a second way to buy green, so
DIALECTED's message carries the⛔ MAINTAINER-ONLYlabel under the #8435 convention, with its own detector and its own three self-test assertions (the detector still reaches its subject; the real message carries the marker; an unmarked offer is rejected).Ablation — the gate must be able to fail
Direction predicted in writing before the run: mutating the #11456 file so it no longer routes through the testkit turns the new gate red with exactly one
DIALECTEDerror naming it, leaves the covered-cell count at 45, and leaves the baseline gate green.Mutation: in
sql-driver-aggregation-conformance.test.ts, the import specifier'./live-dialect-matrix.testkit.js'→'./ablated-not-the-testkit.js'. This reproduces the pre-#11456 state at the level the census reads: still drivesAGGREGATION_CASES, no longer routes through the matrix.Proven on disk by anchored grep counts of the text actually changed, before any result was read — removed text present
0, injected text present1.origin/main)45 covered cell(s)45 covered cell(s)+ dialect line45 covered cell(s)DIALECTED: … sql-driver-aggregation-conformance.test.tsdiffof the baseline census's two runs is empty: the pre-change gate's output is byte-identical on the clean and mutated trees. That is the defect of #12014, reproduced on this tree with a real mutation rather than argued.Restore was verified, not trusted:
trap … EXIT INT TERM, thengit checkout HEAD -- "$TARGET"with an absolute path (never a repo-relative one after acd, and nevergit checkout --alone, which restores from the index the mutation step itself wrote). Thengit hash-objectof the working file returnede488edad7ae0994c9a5a3ed5ff7b890d1ef9087c, equal togit rev-parse HEAD:packages/drivers/driver-sql/src/sql-driver-aggregation-conformance.test.ts, andgit status --porcelainwas empty. The ablation was re-run after the comment-mask swap, since the detector changed.The same red/green is permanent in
--self-test, driven over a synthetic tree by the realdialectAuditrather than a re-implementation of it, along with the stance classifier in all three directions, the ledger reconciling in all three directions, and the non-vacuity guard that fails if the prose-only fixture ever stops being recognisable as a stance in raw text.Verification
Gate union derived, not recalled —
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack(no paths; the script takes the changeset itself). All run atbc203848df, the final commit. Every exit code captured before any pipe; each verdict line below is the gate's own, not$?.check:driver-conformanceOK — 45 covered cell(s), 0 in the DEBT ledger, 0 exempt.+ the dialect line above; self-testOKcheck:agent-test-spelling0 violations — 374 file(s) · 3808 bare -- token(s) …check:cross-package-test-inputsOK: 16 package(s) read outside themselves, all declared …check:entry-guard161 scripts/ file(s) — every entry guard goes through invoked-as.mjs …check:parse-guard160 scripts/ file(s) — every TypeScript parse goes through ts-parse.mjs.check:pnpm-filter-targets135/170 --filter occurrence(s) across 27 file(s) resolve …check-ci-filter-parityOK: all 96 declared cross-package glob(s) (81 unique) are covered …check-cross-package-test-inputsOK: 16 package(s) read outside themselves, all declared …check:nul-bytesOK (scanned 6728 text file(s) … no raw ASCII control bytes).Three families the derivation flagged as artifact rosters co-located with
scripts/— where asilentverdict "is not evidence in EITHER direction" — were run rather than reasoned about:check:i18n-stale-fill,check:pm-dispatch-gates(580 cases pass),check:partof-closing-keyword(28 cases pass). All green.Because this PR now depends on the shared comment mask, its two owning gates were run too:
js-comment-mask --self-test(35 cases pass) andcheck-comment-mask-corpus(5098 files, 0 disagree, 0 unparseable) — the sweep that diffs the mask against@typescript-eslint/parseracross the tree.npx eslint --no-inline-config --format json scripts/check-driver-conformance.mjs— 1 file, 0 errors, 0 warnings.Changeset
None,
skip-changesetlabel. The diff is one repo gate script; no published package changes and there is nothing to release. Graded againstcheck-empty-changeset's own ledger, which is one-directional on exactly this case: an empty changeset satisfies the same gate the label does, produces no CHANGELOG entry (measured), and is uniquely a real input tochangesets/action— thehasChangesets && !hasNonEmptyChangesetsbranch that silently stalled 17.0.0-rc.2 in #4898. It "buys nothing the label does not, and uniquely carries the risk".The label was applied through the whole-set fallback rather than the additive endpoint:
POST /issues/12134/labelsreturned HTTP 403 ("GitHub access is not enabled for this session"), the known per-session outage. So the current set was read (size/l), the union computed, and both written together. Read back after the write: see below.Scope
Held to the declared surface:
scripts/check-driver-conformance.mjsonly.live-dialect-matrix.testkit.tsneeded no widening —dialectCell()already exists and is the marker. No test file is edited, so the two held files (sql-driver-11224-update-stamp-precision.test.ts,sql-driver-11794-richtext-text-family.test.ts) and the three heldsql-driver.tsbands are untouched.Clause ② — no. A census script plus a convention marker; no public surface widens, no shipped accept/reject door moves. What a suite asserts about product behaviour is unchanged.
Generated by Claude Code