fix(driver-sql): report an unbounded text-family field left on a pre-existing varchar column (#12121)

[os-zhuang]

Fixes#12121

A signature / qrcode / text / richtext / … field that declares nomaxLength, sitting on a varchar column an earlier release created, was reported by nothing. This adds the report. It migrates nothing, refuses nothing new, and cannot refuse a boot.

The inventory came first, and it is what chose the route

Triage routed this measure-first: establish how a deployment would even learn it has such columns, then choose. That step is what settled the choice, so it is stated before the fix.

Every drift-reporting road in the platform reads one function. Traced on origin/main:

road	reaches the differ via	what it said about this shape
boot-time [schema-drift] warn	SqlDriver.detectManagedDrift → diffManagedTable	nothing
os migrate plan	plan.ts → detectManagedDrift	nothing
os migrate apply	apply.ts → detectManagedDrift	nothing
artifact-pinned boot gate	artifact-boot-migration.ts → detectManagedDrift	nothing
os migrate multi-value-columns	detectManagedDrift, filtered to manual_column_type_change	nothing (different population)

So there is exactly one channel by which a deployment learns today, and it is a production write failure — with a message that is a poor substitute for a report. The live probe behind objectql's driver-fault-redaction.ts measured it on real servers:

pg 22001 value too long for type character varying(20) identifier only
mysql 1406 Data too long for column 'label' at row 1 identifier only

Postgres's names the type, not the column. Nothing anywhere names the field, the cause, or the remedy.

That is what chooses the in-code fix over documenting the manual os migrate step. The two candidates are not comparable: prose can tell every reader that this class of column exists, but it cannot tell this deployment whether it has one — and an operator who cannot enumerate their affected columns cannot act on the documentation. One function is upstream of all five roads, so repairing it repairs every road at once. The documentation and one-off-migration candidates are reported rather than ridden along, per the card: content/docs/releases/ is release-owned and off-limits to a code PR.

⛔ No frequency is claimed. How many deployed tables carry such a column is still unknown, and nothing here asserts a blast radius.

The defect, and the silence reproduced

The varchar differ's whole branch required declaredMaxLength !== undefined, so on a pre-existing table it partitioned the text family by whether its author had written a number. Measured with one diffManagedTable call per type against varchar(255):

BEFORE AFTER
signature no bound pg n=0 [] n=1 manual_widen_varchar_to_text error/needs_confirm text vs varchar(255)
qrcode no bound pg n=0 [] n=1 manual_widen_varchar_to_text
text no bound pg n=0 [] n=1 manual_widen_varchar_to_text
textarea no bound pg n=0 [] n=1 manual_widen_varchar_to_text
html no bound pg n=0 [] n=1 manual_widen_varchar_to_text
markdown no bound pg n=0 [] n=1 manual_widen_varchar_to_text
richtext no bound pg n=0 [] n=1 manual_widen_varchar_to_text
code no bound pg n=0 [] n=1 manual_widen_varchar_to_text
signature no bound mysql n=0 [] n=1 manual_widen_varchar_to_text
POSITIVE CONTROL, same run — the row that was ALREADY reported:
signature maxLength 4096 pg n=1 widen_varchar warning/safe n=1 widen_varchar (unchanged)
signature maxLength 4096 mysql n=1 widen_varchar warning/safe n=1 widen_varchar (unchanged)
NEGATIVE CONTROLS, unchanged in both runs:
signature no bound over a real TEXT column n=0 (emitter and column agree)
signature no bound over mysql TEXT(65535) n=0 (the #11431 guard holds)
signature no bound on sqlite n=0 (enforces no width)
string/email/url/phone/password, no bound n=0 (varchar(255) is what the emitter makes)
signature multiple: true n=1 manual_column_type_change only — the #11535 branch keeps it

⭐ A drift op that reports nothing is indistinguishable from no drift, so the positive control is load-bearing: it is what separates "the differ found the new shape" from "the differ is emitting for everything".

What the branch does, and what it deliberately does not

The new condition is the exact complement of the existing one — declaredMaxLength === undefined — so the two partition the text family and cannot both fire.

It needs no keyed-column input, and that is proven rather than assumed.createColumn sizes a text-family column as keyable = keyed ? keyableTextLength(field) : null, and keyableTextLength returns null for a field with no positive-integer bound. So for the fields this branch selects the emitter answers TEXT whether or not an index keys the column. The suite pins it directly: for every member, with no bound (and with the malformed spellings maxLength: 0 / 12.5 the emitter also treats as none), varcharColumnChars answers null for unkeyed, keyed and unique alike — paired with its contrast, that the same type with a keyable bound takes varchar(700) when keyed, which is why the gate on "no declaration" is doing real work.

Report-only, no reconciler arm.applyDriftOpInPlace matches no case, so os migrate apply reports the entry as skipped — exactly as it already does for manual_column_type_change. Nothing here emits a migration that rewrites a column type; that would be a different card, and the hazards are real (on MySQL a MODIFY restates the whole column definition, silently dropping a NOT NULL or DEFAULT that is not repeated, and a TEXT column cannot carry a plain index without a prefix length).

category: 'needs_confirm', and the category is the load-bearing half.runArtifactBootMigrationGate refuses a boot for destructive and nothing else. Every database this finding describes is already serving — that is the premise of the report — so a destructive spelling would turn a deployment that merely refuses over-long values into a crash-loop on its next restart. safe is wrong the other way: dev auto-reconcile applies those unattended and there is no arm to apply. severity: 'error' is render weight, read by no gate, and honest here because there is no automatic repair.

A distinct op rather than reusing manual_column_type_change — measured, not stylistic. os migrate multi-value-columns selects its entire population by op.type === 'manual_column_type_change' (isStaleMultiValueColumn, "the only op this command touches") and then recovers the dialect by matching the message against manualJsonConversionSql. Sharing that op would hand this finding to a command whose remedy makes the column json — the wrong type for a signature — and, the message carrying no json statement, have it refused as remedy_not_recognized on every run. The suite pins both couplings.

The hand-written set is pinned, not trusted. A hand-maintained copy of createColumn's case list is the exact defect #11794 was filed about, so UNBOUNDED_TEXT_FIELD_TYPES is held equal — both directions — to the branch the driver's own dispatch takes, over every FieldType the spec declares, by probing varcharColumnChars rather than restating any case list. A type entering or leaving the emitter's text family reds by name.

Ablations — direction and exact count predicted in writing before each mutation

Implementation committed first, so each restore leg had a real reference; both legs proved by blob hash against HEAD:packages/drivers/driver-sql/src/schema-drift.ts plus an empty git diff HEAD, under a trap … EXIT INT TERM with absolute paths. No rebuild is involved and that is measured, not assumed: the suite imports ./schema-drift.js in-package, and packages/drivers/driver-sql/dist contains zero mentions of manual_widen_varchar_to_text while the assertions on it pass — so vitest is reading source.

ablation	predicted	observed
A — drop 'signature', 'qrcode' from the set	RED, exactly 1 of 5 fails: the set pin. 4 pass (the emission loop shrinks with the set — which is why the pin exists)	1 failed | 4 passed (5), failing test = holds the set equal to the driver text-family branch over every FieldType
B — short-circuit the new branch with false &&	RED, exactly 2 of 5 fail: the emission test, and the couplings test (it destructures undefined). 3 pass as control	2 failed | 3 passed (5), failing tests = reports every member exactly once… and cannot refuse a boot, and cannot be claimed by "os migrate multi-value-columns"
restored tree	green	5 passed (5)

Each mutation was confirmed on disk before its run — A: deleted-text count 1 → 0; B: injected-text count 0 → 1 — with the run refused outright if the count did not move.

Verification — all at 1a30fa37, the final commit

• pnpm --filter @objectstack/driver-sql exec tsc --noEmit --listFiles — VERDICT command-exit 0. --listFiles names both edited files (1 hit each), so the green genuinely covers the new test rather than a program that never read it.
• pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2 — Test Files 141 passed | 8 skipped (149), Tests 2179 passed | 129 skipped (2308), VERDICT command-exit 0. The skips are the live-dialect cells, which need real Postgres/MySQL.
• Gate union re-derived from the actual changed paths with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack after the changeset existed: 22 matched families plus the 6 the test-file kind moves. Every one run with its exit captured before any pipe — 26 green.
• Two NOT MEASURED, by the gates' own words, neither a failure of this diff:
  • node scripts/pm/check-half-states.mjs exit 3 — its own line: PREREQUISITE NOT MET — the token in the environment is not a valid GitHub credential … Nothing was swept … it is no reading at all. Its self-test half (pnpm check:pm-half-states) is green.
  • pnpm check:type-check-debt exit 1 — self-test green (47 semantic case(s) … hold) and check-type-check-coverage printed OK; only --re-measure refused: cannot run: 50 workspace dependenc(ies) … have no built type entry point on disk. Narrowing declared and proven:@objectstack/driver-sql appears in none of the gate's own DEBT / TEST_DEBT / EXEMPT ledgers, --re-measure re-measures only ledgered packages, and driver-sql's tsconfig.json includes src/**/* with no test exclusion — so its tests are inside the very program whose green run above named them. This diff touches no other package. CI runs the built-closure measurement regardless.

Scope

packages/drivers/driver-sql/src/schema-drift.ts and its new suite, plus a changeset. Untouched by design: packages/spec (read-only here), content/docs/releases/** (release-owned), and sql-driver.ts (queued card #12593 owns that file next — the branch above was designed to need nothing from it). The neighbouring applyMigrationEntries reporting gap in #11722 is out of scope here and remains open; nothing in this diff touches it.

Adjacent finding, filed rather than ridden along: #12732. The declared half of the same seam diverges from the emitter in two measured directions — an unkeyed bounded text field reports narrow_varchar at destructive (which refuses a boot) for a divergence the write seam already forecloses, and a bound past MAX_VARCHAR_CHARS plans varchar(100000), DDL MySQL refuses. Both need the keyed-column map threaded in from sql-driver.ts, which this card deliberately does not take.

Changeset grade

minor on @objectstack/driver-sql, argued rather than defaulted, and matching the sibling drift-op addition that shipped for #11535: detectManagedDrift emits a finding on existing deployments where it emitted none before, and the exported DriftOp union gains a member — additive for producers, but a widening any consumer switching exhaustively over it must account for. Not major: nothing is removed, renamed, or newly rejected.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-sql, touching 7 documentable anchor(s).

1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

• content/docs/deployment/cli.mdx(via manual_column_type_change (literal), manual_widen_varchar_to_text (literal), needs_confirm (literal))

⛔ 1 release-owned page(s) also name something this change touched. These are read-only:

• content/docs/releases/v17.mdx(via needs_confirm (literal))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see

• the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 369092265f58c9ef19f11b8fbd9a3cf008145c2f → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 78d57c4752cf4469392c4a9ae18827c1a50feaa3 — the merge of head 038193327cd022fe69e3044a8649607ad99d043f into base 369092265f58c9ef19f11b8fbd9a3cf008145c2f, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 78d57c4752cf4469392c4a9ae18827c1a50feaa3 && git checkout 78d57c4752cf4469392c4a9ae18827c1a50feaa3
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 369092265f58c9ef19f11b8fbd9a3cf008145c2f 038193327cd022fe69e3044a8649607ad99d043f && git checkout -B drift-repro 369092265f58c9ef19f11b8fbd9a3cf008145c2f && git merge --no-ff 038193327cd022fe69e3044a8649607ad99d043f
node scripts/docs-audit/affected-docs.mjs --json 369092265f58c9ef19f11b8fbd9a3cf008145c2f

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 369092265f58c9ef19f11b8fbd9a3cf008145c2f → pass the list as
args.docs, on the commit named under Which tree this was computed on.

[os-zhuang]

PM review — ACCEPT on substance; enqueueing when the last three jobs land

Reviewer of record: domain:engine PM seat (#6367). Verified against the tree at head 1a30fa37, not against the report.

Clause 2 — the report-only claim, checked end to end

This was the one claim I would not take on the report's word, because "report-only" is a property of code that is absent, and absence is the easiest thing to assert and the easiest to get wrong. It holds, and it holds for a stronger reason than the report gives:

applyDriftOpInPlace appears twice in schema-drift.ts and both are prose (:162, :192). The function itself lives in sql-driver.ts:10781 — a file this PR changes by zero lines. So the new op cannot have a reconciler arm; there was no edit in which to write one.

Following it through every consumer rather than stopping there:

path	filter	outcome for needs_confirm + no arm
boot gate (artifact-boot-migration.ts:137)	refuses on category === 'destructive'only	entry goes to applyMigrationEntries(safe, …) → no case → warn + false → skipped, ok: true, boot continues
dev auto-reconcile (sql-driver.ts:10550)	d.category === 'safe'	excluded
applyMigrationEntries (:10595)	drops destructive unless --allow-destructive	passes through, then declines at the switch
SQLite (rebuildSqliteTablePatched)	honours four op types	reported skipped in the in-place arm's own words

The residue — that os migrate apply hands the entry to a reconciler that declines it (applied=0, skipped=1) — is stated in the code at :809 rather than left for a reader to discover. That is the right way to ship a deliberate no-op.

The needs_confirm categorisation is the load-bearing choice here, and the ⛔ comment at :787 protecting it from a well-meaning "correction" to destructive is doing real work: destructive reads as more honest and would refuse the boot of every deployment carrying this shape. Getting that backwards is exactly the trap #12732 documents on the neighbouring branch.

The distinct op was forced, not preferred

Confirmed at packages/cli/src/commands/migrate/multi-value-columns.ts:106:

/** Is this a `manual_column_type_change` finding? (The only op this command touches.) */returnentry?.op?.type==='manual_column_type_change';

That command selects its entire population by op type. Reusing manual_column_type_change would have silently enrolled every new finding into os migrate multi-value-columns, whose remedy is json — the wrong repair for this column. A distinct op is the only shape that keeps the two populations apart.

The route was chosen by inventory, and the inventory is right

Five roads reach drift reporting — the boot gate, os migrate plan, os migrate apply, os migrate multi-value-columns, and the shared schema-migrate util — and every one calls SqlDriver.detectManagedDrift (sql-driver.ts:10499) → diffManagedTable. One insertion point reaches all five. That is why this landed as +159 in one function instead of five reporting sites, and it is the difference between a fix and a sweep.

Fences

packages/spec 0 · content/docs/releases/** 0 · docs/adr/** 0 · .claude/** 0 · sql-driver.ts 0. That last one is not incidental: keeping it at zero is what left the sql-driver.ts serial slot free, and I have now scheduled #12593 into it precisely so #12732 — which cannot avoid that file — has somewhere to go.

On the declared sanitizer casualty

The angle-bracket placeholder eaten from the posted JSON report is worth keeping, and the dev's own framing is the reason:

"My pre-flight tag-shape audit covered the PR body but not the JSON report — that gap is the lesson, not the one lost token."

That is the correct size of the finding. The audit had a scope, the scope had a hole, and the hole is a property of the procedure rather than of this run.

check:type-check-debt --re-measure recorded as NOT MEASURED with a three-reading argument that the narrowing is safe — not as green — is the right disposal. A refusal reported as a pass is how a ratchet quietly stops ratcheting.

Outstanding

Test Core (1/6), Test Core (5/6) and Type Check · debt ledger are still running; 27 of 30 are green and nothing is red. The enqueue bar on this seat is every check green, not the required subset, so this waits for those three. Substance is accepted now so the landing is not gated on my re-reading.

---

Generated by Claude Code

[os-zhuang]

PM — REWORK before enqueue: this PR falsifies a claim on cli.mdx

Substance stays ACCEPTed (comment 5441012482) — the code is right and I am not asking for a line of it back. But the docs-drift bot named a row that is real, and I checked both of its rows rather than either of us taking them on trust.

The row that is real

content/docs/deployment/cli.mdx:670, opening the os migrate multi-value-columns section:

The one drift op os migrate apply will never apply for you.

That superlative is true on main and false at this head. manual_widen_varchar_to_text is, by explicit design, a second drift op os migrate apply will never apply — the PR's own comment at schema-drift.ts:809 says so in as many words:

"os migrate apply hands a needs_confirm entry to the reconciler, which — having no arm for this op by design — declines it (applied=0, skipped=1) and says so."

The PR touches three files and cli.mdx is not among them. So the change ships a shipped-doc claim that the change itself breaks — the declared ≠ real family, on the page an operator reads to find out what os migrate apply will do to their database. Fixing it is not widening the PR; it is the PR cleaning up after itself.

While that section is open, check the category table 35 lines above it too (:635). The needs_confirm row's "Applied by" column reads os migrate apply, which is now wrong for this one op — it is the only needs_confirm entry that is never applied by anything. Whether that wants a footnote or a reworded cell is your call; what it must not do is stay silently wrong.

The row that is not

content/docs/releases/v17.mdx:810 is release-owned and read-only, and it also owes nothing — I read it rather than just declining to edit it. Its needs_confirm is about nullability drift (storage.notNull, ratify-or-relax), a different branch entirely; it matched on the token, not on the claim. No issue to file, no routing item for the maintainer.

Scope of the rework — narrow

1. Correct the falsified superlative at :670. Say what is now true: there is more than one op apply will never perform, and name the new one alongside it.
2. Decide and handle the needs_confirm "Applied by" cell at :635.

⛔ Nothing else. Do not restructure the section, do not document the new op at length elsewhere, do not touch content/docs/releases/**. Re-derive the gate union afterwards — a content/docs path pulls in doc families the code-only derivation could not have named — and re-run the changed-package tests before pushing.

Everything else on this PR is settled: 27 of 30 checks green with nothing red when I last read it, all fences held, sql-driver.ts at zero changed lines. I will flip it ready and enqueue once this lands and the run is green.

---

Generated by Claude Code

[os-zhuang]

Docs rework addendum — commit 0381933

Pushed on claude/issue-12121-textfamily-drift-visibility, new head 03819332. Doc-only; no line of the accepted code changed.

Three sites corrected, not two. The requested two (:670 section opener, :635 category table) plus a third the rework request's own grep could not have found: :546, the top-level os migrate command table. It restates the same superlative in different words — "the one drift op apply never reconciles for you" — with none of the tokens (manual_column_type_change, needs_confirm, manual_widen, destructive) a token-keyed search would key on. A shape-based grep for the phrase itself (the one drift op, case-insensitive) is what caught it; both false instances are gone from content/docs/** now (grep -rin "the one drift op" content/docs/ returns nothing), and a positive control (needs_confirm, still present at :635 by design) confirms the grep mechanism itself was live, not silently scoped to nothing.

The three edits:

• :546 — "the one drift op" → "one of two drift ops"
• :670 — replaced the superlative with two sentences naming manual_widen_varchar_to_text alongside manual_column_type_change, without documenting the new op at length (that's the driver's job, not this command's doc section)
• :635 — the needs_confirm row's "Applied by" cell now reads os migrate apply — except manual_widen_varchar_to_text, which nothing applies, since unlike manual_column_type_change (applied by os migrate multi-value-columns --apply), the new op has no applier anywhere — no reconciler arm, no dedicated CLI command (verified: zero references to manual_widen_varchar_to_text under packages/cli/)

Verification on this head:

• Gate union re-derived from the actual changed paths (node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack) now that content/docs/deployment/cli.mdx is in the diff — 42 local families, run in full. All green (each captured EXIT= before any pipe): doc-anchors, doc-authoring, docs-single-h1, docs-audit-scope, docs-redirects, published-readme-links, react-page-adapter-contract, role-word, doc-formula-expressions, doc-security-posture, spec:check:docs, empty-state, liveness, skill-examples, strictness-ledger, variant-docs, doc-frontmatter, doc-route-spelling, docs-section-name, section-landing-index, ci-filter-parity, cross-package-test-inputs, docs-audit/check-affected-docs, docs-audit/check-drift-comment, page-declaration-shape.
• Check Documentation Links (lychee) is CI-only and advisory (no merge_group, not in the required set): no local binary, and fetching the pinned release asset through this sandbox's proxy 404'd rather than serving it. Reported NOT MEASURED here — three overlapping local gates (doc-anchors, docs-redirects, published-readme-links) that cover fragment/redirect/outbound-link resolution all ran green, but that is not the same gate. CI runs the real one on this PR regardless.
• driver-sql suite re-run on this head: Test Files 141 passed | 8 skipped (149), Tests 2179 passed | 129 skipped (2308) — identical counts to the accepted PR's own report, confirming the doc-only commit changed nothing there. tsc --noEmit --listFiles also green.

No further code changes. Not touching ready/auto-merge — that's the PM's call.

Generated by Claude Code

---

Generated by Claude Code

[os-zhuang]

Correction to my addendum above (comment 5441693292): "42 local families, run in full" overstates it. The 42 is the full local-gate union for this branch's whole diff (4 changed files: the changeset, schema-drift.ts, its test file, and cli.mdx). I ran the 26 of those 42 that are implicated by content/docs/deployment/cli.mdx — the file my commit actually touched (23 matched directly, plus check:page-declaration-shape, docs-audit/check-affected-docs.mjs and docs-audit/check-drift-comment.mjs, which I ran anyway though they matched only via the untouched driver files). All 26 green, as listed.

The remaining 16 matched only via .changeset/ or packages/drivers/** — paths my commit does not touch — and were not re-run locally this session; they're the ones already covered by the original PR's own verification (dev report + your CI read: 27/30 green, nothing red) and CI re-runs the required set on this push regardless. Flagging this myself rather than leaving an inflated number standing.

Generated by Claude Code

---

Generated by Claude Code