diff --git a/.changeset/richtext-code-text-family-emission.md b/.changeset/richtext-code-text-family-emission.md new file mode 100644 index 0000000000..185d67f4a7 --- /dev/null +++ b/.changeset/richtext-code-text-family-emission.md @@ -0,0 +1,11 @@ +--- +"@objectstack/driver-sql": patch +--- + +A `richtext` field now takes an unbounded TEXT column instead of knex's `varchar(255)`, so an ordinary rich-text body over 255 characters can be written (#11794). `createColumn`'s text-family case listed `text` / `textarea` / `html` / `markdown`; `richtext` — the third member of the spec's own "Rich Content" grouping in `field.zod.ts` — was in neither that case nor `JSON_COLUMN_TYPES`, so it fell through to the catch-all's `table.string(name)`. Measured at 1000 characters on live MySQL 8.0.46 and Postgres 16: before this change the write was refused by the server (`ER_DATA_TOO_LONG` under `STRICT_TRANS_TABLES`, `22001 value too long for type character varying(255)`) while the same body in a `markdown` field on the same table was accepted; after it, the column reads back as `text` from `information_schema` on both and the value round-trips byte-identically. `code` moves with it for the same reason. + +Membership is now decided by a stated, measured test instead of the hand-maintained case list that let one member of a three-member spec group diverge in the first place: a type may take an unbounded TEXT column exactly when the **write seam** enforces its declared `maxLength`, which is the invariant `schema-drift.ts` already rests on ("A TEXT column refuses nothing a `maxLength` allows … the bound is enforced at the write seam"). objectql's record-validator applies its `max_length` branch to `text` / `textarea` / `email` / `url` / `phone` / `password` / `markdown` / `html` / `richtext` / `code` and to nothing else, so both moved types keep a field-named ADR-0112 refusal for an over-declared value and the physical surface is restored to the declared contract rather than widened past it. The set of types that take an unbounded column when unkeyed is pinned as a whole, so the next addition has to be stated on purpose. + +`signature` and `qrcode` are **not** moved, deliberately and against the first reading of this defect. Their stored value is the author's own and routinely far past 255 characters (a data-URI PNG), so `varchar(255)` refuses ordinary values for them too — but the record-validator has no `max_length` branch for either, so an unbounded column would accept values a declared `maxLength` forbids: over-accepting in place of under-accepting, which is a physical surface wider than the contract. They stay bounded until the write seam can bound them, and the live-dialect suite asserts that refusal out loud rather than leaving it undocumented. + +The #11374 keyed-and-bounded rule applies to the two new members unchanged: a keyed, bounded `richtext` / `code` column is still emitted as `varchar(maxLength)` so a declared index can key it on MySQL, and a keyed but unbounded one still gets the named `explainUnkeyableTextColumn` refusal rather than a silently weaker constraint. Nothing about existing tables changes — `createColumn` runs on `CREATE TABLE` and `ALTER TABLE ADD COLUMN`, so the column it sizes is always empty. diff --git a/packages/drivers/driver-sql/src/sql-driver-11794-richtext-text-family.test.ts b/packages/drivers/driver-sql/src/sql-driver-11794-richtext-text-family.test.ts new file mode 100644 index 0000000000..b3f609f23f --- /dev/null +++ b/packages/drivers/driver-sql/src/sql-driver-11794-richtext-text-family.test.ts @@ -0,0 +1,265 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #11794 — `richtext` joins its declared "Rich Content" siblings in the TEXT + * family, taking an unbounded column instead of knex's varchar(255). + * + * ## The defect + * + * `createColumn`'s text-family case listed `text` / `textarea` / `html` / + * `markdown`. `richtext` — the third member of the spec's own "Rich Content" + * grouping (`field.zod.ts`) — was not in it, and not in `JSON_COLUMN_TYPES` + * either, so it fell through to the catch-all's `table.string(name)`: + * varchar(255). That width is a hard cap on both enforcing dialects, so an + * ordinary rich-text body over 255 characters was REFUSED at write time + * (measured at 1000 characters on live MySQL 8.0.46 — `ER_DATA_TOO_LONG` + * under `STRICT_TRANS_TABLES` — and Postgres 16 — `22001`) while the same + * body in a `markdown` field on the same table was accepted. + * + * ## Which types moved, and the test that decided it + * + * `code` moved with `richtext`. `signature` and `qrcode` did NOT, and that is + * the load-bearing half of this file rather than an omission. + * + * An unbounded TEXT column is correct for a type exactly when the WRITE SEAM + * enforces that type's declared `maxLength` — the invariant `schema-drift.ts` + * already states ("A TEXT column refuses nothing a `maxLength` allows … the + * bound is enforced at the write seam"). objectql's record-validator applies + * its `max_length` branch to `text` / `textarea` / `email` / `url` / `phone` / + * `password` / `markdown` / `html` / `richtext` / `code` — and to no other + * type. Measured: a `maxLength: 64` field of each of those refuses a + * 100-character value; the same field declared `signature` or `qrcode` + * ACCEPTS it. So for those two an unbounded column would accept values the + * declaration forbids — a physical surface WIDER than the contract, where + * `richtext` and `code` are a restoration of it. Their own defect (a data-URI + * signature capped at 255) is real and is asserted here as an open one, so + * this file records the state rather than hiding it. + * + * ## What each block is worth + * + * The SQLite blocks run everywhere (Test Core included) and read the PHYSICAL + * column type back from the PRAGMA (`columnInfo()`), never the emitter. The + * live cells are the enforcing half: the same table on a real MySQL / + * Postgres, column types read from information_schema, a 1000-character body + * accepted and round-tripped — made non-vacuous by the control write, where + * the SAME oversized value into a column this change deliberately left at + * varchar(255) is refused BY THE SERVER, proving the cell enforces declared + * widths in this very run. + */ + +import { describe, it, expect, afterEach } from 'vitest'; +import { FieldType } from '@objectstack/spec/data'; +import { SqlDriver } from '../src/index.js'; +import { MYSQL_CELL, PG_CELL, dialectCell, declareDialectCell } from './live-dialect-matrix.testkit.js'; + +const T = 'os11794_text_family'; + +/** The two this card moves, their siblings, and the stay-put controls. */ +const FIELDS = { + // Moved by #11794: varchar(255) → TEXT. + body_rich: { type: 'richtext' }, + body_code: { type: 'code' }, + // Positive controls: TEXT before and after this change — the grouping was + // already honoured for two of the three Rich Content members. + body_md: { type: 'markdown' }, + body_html: { type: 'html' }, + // Measured and deliberately NOT moved: no write seam enforces their + // `maxLength`, so TEXT would accept what the declaration forbids. + body_sig: { type: 'signature' }, + body_qr: { type: 'qrcode' }, + // Negative controls: the catch-all and the string family. + c_string: { type: 'string' }, + c_select: { type: 'select' }, + c_color: { type: 'color' }, + c_secret: { type: 'secret' }, +}; + +const OPTS = { bypassTenantAudit: true } as any; + +/** A rich-text body nobody would call exotic — four times the old cap. */ +const LONG_BODY = `

${'a rich-text body well past the old varchar(255) cap — '.repeat(20)}

`; + +const MOVED = ['body_rich', 'body_code'] as const; +const SIBLINGS = ['body_md', 'body_html'] as const; +const NOT_MOVED = ['body_sig', 'body_qr', 'c_string', 'c_select', 'c_color', 'c_secret'] as const; + +/** + * Every FieldType that takes an UNBOUNDED column when no index keys it — + * pinned as a SET rather than left to the switch. + * + * The root cause this card names is that the case list is hand-maintained, so + * one member of a three-member spec group diverged from the other two without + * anything going red. A membership pin is what makes that impossible: adding a + * type to `createColumn`'s text family, or to `JSON_COLUMN_TYPES`, fails here + * until someone states the new membership on purpose. + */ +const UNBOUNDED_UNKEYED = [ + // text family (`createColumn`) — every member must satisfy the write-seam + // invariant in this file's header. + 'text', 'textarea', 'html', 'markdown', 'richtext', 'code', + // JSON columns and the virtual/non-varchar types: not a varchar either, for + // reasons that have nothing to do with this card. + 'multiselect', 'checkboxes', 'tags', 'composite', 'repeater', 'record', 'json', + 'location', 'address', 'vector', 'image', 'file', 'avatar', 'video', 'audio', + 'formula', 'number', 'currency', 'percent', 'rating', 'slider', 'progress', + 'summary', 'boolean', 'toggle', 'date', 'datetime', 'time', +].sort(); + +/** TEXT and not any varchar — `longtext`/`mediumtext` would satisfy it too. */ +const isTexty = (t: unknown) => /text/i.test(String(t)) && !/varchar/i.test(String(t)); + +type ColumnInfo = Record; + +describe('richtext joins the TEXT family (#11794) — physical shape on SQLite', () => { + let driver: SqlDriver; + + afterEach(async () => { + await driver?.disconnect().catch(() => {}); + }); + + it('lands richtext/code as TEXT beside markdown/html — and moves nothing else', async () => { + driver = new SqlDriver(dialectCell('sqlite').config()); + await driver.initObjects([{ name: T, fields: FIELDS }]); + // The PRAGMA, not the emitter: knex's columnInfo() reads table_info. + const info: ColumnInfo = await (driver as any).knex(T).columnInfo(); + + for (const moved of MOVED) { + expect(isTexty(info[moved]?.type), `${moved} landed ${String(info[moved]?.type)}`).toBe(true); + } + for (const sibling of SIBLINGS) { + expect(isTexty(info[sibling]?.type), `${sibling} landed ${String(info[sibling]?.type)}`).toBe( + true, + ); + } + for (const still of NOT_MOVED) { + expect( + /varchar/i.test(String(info[still]?.type)), + `${still} landed ${String(info[still]?.type)}`, + ).toBe(true); + expect(Number(info[still]?.maxLength)).toBe(255); + } + }); + + it('round-trips a >255-character richtext body byte-identically', async () => { + driver = new SqlDriver(dialectCell('sqlite').config()); + await driver.initObjects([{ name: T, fields: FIELDS }]); + expect(LONG_BODY.length).toBeGreaterThan(255); + await driver.create(T, { id: 'r1', body_rich: LONG_BODY, body_md: LONG_BODY }, OPTS); + const [row] = await driver.find(T, { where: { id: 'r1' } }, OPTS); + expect(row.body_rich).toBe(LONG_BODY); + expect(row.body_md).toBe(LONG_BODY); // the sibling that always worked + }); + + it('pins the whole unbounded-when-unkeyed SET, so the case list cannot drift again', () => { + driver = new SqlDriver(dialectCell('sqlite').config()); + const mirror = (type: string) => + (driver as any).varcharColumnChars({ type }, undefined) as number | null; + const types = FieldType.options as readonly string[]; + expect(types.length).toBeGreaterThan(40); // the registry really was read + const unbounded = types.filter((t) => mirror(t) === null).sort(); + expect(unbounded.length).toBeGreaterThan(20); // and the filter really matched + expect(unbounded).toEqual(UNBOUNDED_UNKEYED); + // The card's minimum, spelled out: the spec's three-member "Rich Content" + // group is whole again. + for (const t of ['markdown', 'html', 'richtext']) expect(mirror(t)).toBeNull(); + // And the two that measured as wideners stay bounded. + for (const t of ['signature', 'qrcode']) expect(mirror(t)).toBe(255); + }); + + it('keeps #11374 keyed-and-bounded semantics for the new members', () => { + driver = new SqlDriver(dialectCell('sqlite').config()); + const mirror = (field: any, keyed?: { unique: boolean }) => + (driver as any).varcharColumnChars(field, keyed) as number | null; + // Unkeyed: TEXT, bound or not — a column no index keys on gains nothing + // from a width. + expect(mirror({ type: 'richtext' })).toBeNull(); + expect(mirror({ type: 'code', maxLength: 64 })).toBeNull(); + // Keyed and bounded: varchar(maxLength) — the #11374 rule, so a declared + // index on a bounded code field still keys on MySQL. + expect(mirror({ type: 'code', maxLength: 64 }, { unique: true })).toBe(64); + // Keyed and unbounded: still TEXT — MySQL then refuses the key BY NAME + // (explainUnkeyableTextColumn), never a silently weaker constraint. + expect(mirror({ type: 'richtext' }, { unique: true })).toBeNull(); + }); +}); + +// ── The half only an enforcing dialect can measure ────────────────────────── + +for (const liveCell of [PG_CELL, MYSQL_CELL]) { + declareDialectCell(liveCell, 'richtext TEXT family (#11794)', (cell) => { + describe(`richtext TEXT family on live ${cell.label} (#11794)`, () => { + let driver: SqlDriver; + + afterEach(async () => { + await driver?.execute(`drop table if exists ${T}`).catch(() => {}); + await driver?.disconnect().catch(() => {}); + }); + + it('accepts a >255-char richtext body, and the column really is TEXT (information_schema)', async () => { + driver = new SqlDriver(cell.config()); + await driver.execute(`drop table if exists ${T}`).catch(() => {}); + await driver.initObjects([{ name: T, fields: FIELDS }]); + + // information_schema.columns, not the emitter: that is what knex's + // columnInfo() reads on both of these dialects. + const info: ColumnInfo = await (driver as any).knex(T).columnInfo(); + for (const moved of MOVED) { + expect(isTexty(info[moved]?.type), `${moved} landed ${String(info[moved]?.type)}`).toBe( + true, + ); + } + for (const still of NOT_MOVED) { + expect( + /varchar|character varying/i.test(String(info[still]?.type)), + `${still} landed ${String(info[still]?.type)}`, + ).toBe(true); + expect(Number(info[still]?.maxLength)).toBe(255); + } + + // The write this card is about: refused before this change + // (ER_DATA_TOO_LONG / 22001), accepted now, byte-identical back. + await driver.create( + T, + { id: 'r1', body_rich: LONG_BODY, body_code: LONG_BODY, body_md: LONG_BODY }, + OPTS, + ); + const [row] = await driver.find(T, { where: { id: 'r1' } }, OPTS); + expect(row.body_rich).toBe(LONG_BODY); + expect(row.body_code).toBe(LONG_BODY); + + // Non-vacuity control: the SAME oversized value into a column this + // change deliberately left at varchar(255) is refused BY THE SERVER. + // Without this, a mis-provisioned lenient session (MySQL without + // STRICT_TRANS_TABLES) would pass the acceptance above while + // measuring nothing. + const refusal = await driver + .create(T, { id: 'r2', c_color: LONG_BODY }, OPTS) + .then(() => null) + .catch((e: unknown) => e); + expect(refusal).toBeInstanceOf(Error); + const said = `${String((refusal as { code?: string })?.code ?? '')} ${String((refusal as Error).message)}`; + expect(said).toMatch(/ER_DATA_TOO_LONG|22001|too long/i); + }); + + it('records the STILL-OPEN half: an oversized signature is refused by the server', async () => { + // ⛔ Not a wish and not a quarantine — the current, deliberate state. + // `signature` stays varchar(255) because nothing enforces its declared + // `maxLength` at the write seam, so TEXT would accept what the + // declaration forbids. This asserts the cost of that choice out loud: + // a data-URI signature IS refused today. When the write seam gains a + // bound for it, this test is what turns red and gets updated. + driver = new SqlDriver(cell.config()); + await driver.execute(`drop table if exists ${T}`).catch(() => {}); + await driver.initObjects([{ name: T, fields: FIELDS }]); + const dataUri = `data:image/png;base64,${'A'.repeat(1000)}`; + const refusal = await driver + .create(T, { id: 's1', body_sig: dataUri }, OPTS) + .then(() => null) + .catch((e: unknown) => e); + expect(refusal).toBeInstanceOf(Error); + const said = `${String((refusal as { code?: string })?.code ?? '')} ${String((refusal as Error).message)}`; + expect(said).toMatch(/ER_DATA_TOO_LONG|22001|too long/i); + }); + }); + }); +} diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index 6b8cc51c13..0114770fd7 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -13385,6 +13385,8 @@ export class SqlDriver implements IDataDriver { case 'textarea': case 'html': case 'markdown': + case 'richtext': + case 'code': return keyed ? this.keyableTextLength(field) : null; // Virtual — `createColumn` returns without emitting anything. case 'formula': @@ -13679,7 +13681,50 @@ export class SqlDriver implements IDataDriver { case 'text': case 'textarea': case 'html': - case 'markdown': { + case 'markdown': + // #11794: `richtext` and `code` join the text family, and membership is + // decided by a MEASURED test rather than by "this type's values look + // long". + // + // ## What makes an unbounded TEXT column correct for a type + // + // That the WRITE SEAM enforces the type's declared `maxLength` — the + // invariant the rest of this driver already rests on, stated in + // `schema-drift.ts` in as many words: "A TEXT column refuses nothing a + // `maxLength` allows … the bound is enforced at the write seam." So the + // question is not whether a value can be long, it is whether the + // declaration still binds once the column stops binding. + // + // objectql's record-validator applies its `max_length` / `min_length` + // branch to exactly `text` / `textarea` / `email` / `url` / `phone` / + // `password` / `markdown` / `html` / `richtext` / `code`. Both new + // members are inside that list — measured, not read off it: a + // `maxLength: 64` field of each type refuses a 100-character value with + // a field-named ADR-0112 envelope, before any column is reached. So + // moving them here RESTORES the declared contract (any string, as + // `valueSchemaFor` says) instead of widening past it. + // + // `richtext` is the headline member: the spec groups `markdown` / `html` + // / `richtext` together as "Rich Content" (`field.zod.ts`) and two of + // the three already landed here — the third fell through to the + // catch-all's `table.string(name)`, knex's varchar(255), so an ordinary + // rich-text body over 255 characters was refused by both enforcing + // dialects while the same body in a `markdown` field on the same table + // was accepted. Measured at 1000 characters on live MySQL 8.0.46 + // (`ER_DATA_TOO_LONG` under `STRICT_TRANS_TABLES`) and Postgres 16 + // (`22001`). `code` is the same defect on the same evidence — a code + // editor's contents, refused identically on both dialects. + // + // ⛔ `signature` and `qrcode` are STRING_VALUE_TYPES members whose stored + // value is also the author's own and also routinely far past 255 + // characters (field-zoo writes a data-URI PNG for `signature`), and they + // are deliberately NOT here — see the catch-all's note. The validator + // branch above does not list them, so nothing enforces their declared + // `maxLength` anywhere: for them an unbounded TEXT column would accept + // values the declaration forbids, which is a widening of the physical + // surface past the contract rather than a restoration of it. + case 'richtext': + case 'code': { // #11374: a text-family column that some declared index KEYS ON is // emitted as `varchar(maxLength)` rather than TEXT, whenever the field // declared a bound this dialect can key on. @@ -13828,14 +13873,31 @@ export class SqlDriver implements IDataDriver { // (#field-zoo). Everything else is a plain string. // // ⛔ The third branch #11431 leaves alone. This is the CATCH-ALL, and - // what lands in it is precisely the set of types whose stored value is - // NOT the declared value: `secret` persists an opaque `sys_secret` - // ref rather than the credential it was given (ADR-0100), and - // `select` / `radio` / `checkboxes` / `code` / `tree` store option - // machine names or ids. Sizing any of those from the author's - // `maxLength` would size the wrong string. A type that genuinely wants - // the bound belongs in the string-family case above, named — never - // acquired by falling through to here. + // MOST of what lands in it is the set of types whose stored value is + // NOT the declared value, or is short by construction: `secret` + // persists an opaque `sys_secret` ref rather than the credential it was + // given (ADR-0100), `select` / `radio` / `checkboxes` / `tree` store + // option machine names or ids, and `color` holds a color code. Sizing + // any of those from the author's `maxLength` would size the wrong + // string. (`code` used to be mis-listed here among the option-valued + // ones — measured in field-zoo it stores the editor's contents + // verbatim, which is why #11794 moved it to the text family above.) + // + // ⚠️ `signature` and `qrcode` are here for a DIFFERENT reason, and it is + // an OPEN DEFECT rather than a design. Their stored value IS the + // declared value and it is routinely far past 255 characters — a + // data-URI PNG for `signature` — so varchar(255) refuses ordinary + // authored values on both enforcing dialects, exactly the way it did + // for `richtext`. #11794 measured them and left them here anyway, + // because NOTHING enforces their declared `maxLength`: the + // record-validator's `max_length` branch does not list them, so an + // unbounded TEXT column would trade an under-accepting column for an + // over-accepting one — a physical surface wider than the contract. + // They need an enforced bound before they can move; see the text-family + // case above for the invariant that decides it. + // + // A type that genuinely wants the bound belongs in the string-family + // case above, named — never acquired by falling through to here. col = JSON_COLUMN_TYPES.has(type) ? table.json(name) : table.string(name); }