diff --git a/.changeset/driver-sql-keyed-text-maxlength.md b/.changeset/driver-sql-keyed-text-maxlength.md new file mode 100644 index 0000000000..a7339c01e6 --- /dev/null +++ b/.changeset/driver-sql-keyed-text-maxlength.md @@ -0,0 +1,15 @@ +--- +"@objectstack/driver-sql": minor +--- + +**Fix:** a text-family field that a declared index keys on is emitted as `varchar(maxLength)` instead of an unbounded `TEXT`, so the index MySQL previously refused can actually be created (#11374). + +`createColumn` mapped the whole text family (`text` / `textarea` / `html` / `markdown`) to an unbounded `TEXT`, ignoring the field's own declared `maxLength`. MySQL refuses a `TEXT`/`BLOB` column in a key without a prefix length, and the two halves of schema-sync fail *separately*: the `CREATE TABLE` succeeds, then `ALTER TABLE … ADD [UNIQUE] INDEX` fails with `ER_BLOB_KEY_WITHOUT_LENGTH`. The table therefore lands on disk **without the constraint it declared**, and the object stays registered-but-broken. Measured on a live MySQL 8.0.46: **36 of the 44 platform objects** failed schema-sync this way, so a stack whose `default` datasource is MySQL could not stand up its own schema — the dev-admin seed never landed and first sign-in returned `401 INVALID_EMAIL_OR_PASSWORD`. Honouring the declared bound takes that to **12**. + +**The bound is the field's own `maxLength` — nothing is invented.** `schema-drift.ts` already treated `varchar(field.maxLength)` as the expected physical shape of a bounded field (its `widen_varchar` / `narrow_varchar` ops say so in as many words); this is the emitter finally agreeing with the differ. On MySQL that removes a permanent destructive drift finding: `columnInfo()` reports `maxLength: 65535` for a `TEXT` column, so every bounded text field already reported `narrow_varchar` ("metadata caps at 32 chars but the column allows 65535") against a column the driver itself had created. + +**Scope, both halves load-bearing.** The bound is emitted only for a column some declared index **keys on** — a non-indexed `Field.text({ maxLength: 65000 })` stays `TEXT`, because `varchar(65000)` on utf8mb4 is 260000 bytes and would blow MySQL's 65535-byte row limit, turning a working table into an un-creatable one. And only where the bound is **usable as a key part**: `maxLength` absent, or wider than 768 characters (3072 index bytes ÷ 4 bytes per utf8mb4 character — measured: `varchar(768)` takes a unique index, `varchar(769)` is refused with `ER_TOO_LONG_KEY`), leaves the column `TEXT` and the index refused with a message naming the field and the declaration that fixes it. + +**⚠️ Graded `minor`, not `patch`: this changes declared behaviour on newly created tables.** A keyed bounded text column now enforces its declared length where the dialect enforces `varchar` (Postgres and MySQL), so a write longer than `maxLength` that previously landed in an unbounded `TEXT` is now refused — under `STRICT_TRANS_TABLES`, with `ER_DATA_TOO_LONG`. That is the declaration becoming enforced rather than a new restriction, and it is exactly what makes the column indexable, but it is a behaviour change and is named here as one. **Existing tables are unaffected**: schema-sync is additive and never rewrites a column that is already present. + +**A prefix index is deliberately NOT substituted for an unkeyable column.** For an ordinary index that would be a transparent access-path choice, but for a `UNIQUE` one it silently replaces the declared constraint with a stricter one — uniqueness of the *prefix*. Measured on MySQL 8.0.46 with `UNIQUE KEY (token(191))` and two distinct 200+ character tokens sharing their first 191 characters: the second insert was rejected with `ER_DUP_ENTRY` **even though the tokens differ**. On `sys_session.token` that is a valid sign-in refused as a duplicate. The refusal an operator can read is strictly better than a constraint that quietly means something else. diff --git a/content/docs/protocol/objectql/types.mdx b/content/docs/protocol/objectql/types.mdx index fd8dd0b4b3..21f64f7a53 100644 --- a/content/docs/protocol/objectql/types.mdx +++ b/content/docs/protocol/objectql/types.mdx @@ -77,10 +77,35 @@ company_name: ``` **Database mapping:** -- SQL driver: `TEXT` on every dialect. `maxLength` is enforced by record - validation, not by the column type — the DDL does not read it. +- SQL driver: `TEXT` on every dialect — **except when a declared index keys the + column**, where it is `VARCHAR(maxLength)` instead. Both conditions are + required: the field declares a `maxLength` of **768 or less** (the widest key + part utf8mb4 allows — MySQL's 3072-byte index limit ÷ 4 bytes per character), + **and** some declared index keys on it (field-level `unique`, or an entry in + the object's `indexes[]`). So the `company_name` field above stays `TEXT` + unless an index names it, and a keyed field declaring `maxLength: 1024` stays + `TEXT` too. - MongoDB: `String` + +**Why the bound follows the index.** MySQL refuses a `TEXT`/`BLOB` column in a +key without a prefix length, so an unbounded keyed column makes its own index +un-creatable — the `CREATE TABLE` succeeds and the `ALTER TABLE … ADD INDEX` +fails, leaving the table without the constraint it declared. Bounding an +*unkeyed* column would buy nothing and cost something: `TEXT` is stored +off-page, while a wide `VARCHAR` counts against MySQL's 65535-byte row limit. + +When a keyed column cannot be bounded, the driver **refuses the index** and +names the field, rather than silently substituting a prefix index — a +prefix-`UNIQUE` constrains the prefix rather than the value, so it rejects two +different values that happen to share one. + +`maxLength` is enforced by record validation on **every** field. On a keyed +column it is *additionally* enforced by the column type, so PostgreSQL and +MySQL refuse an over-length write at the database as well (SQLite does not +enforce `VARCHAR` length). + + **UI rendering:** ```html @@ -104,7 +129,8 @@ description: ``` **Database mapping:** -- SQL driver: `TEXT` +- SQL driver: `TEXT` — or `VARCHAR(maxLength)` when a declared index keys the + column, on the same two conditions as the `text` type above. - MongoDB: `String` **UI rendering:** @@ -129,7 +155,8 @@ bio: ``` **Database mapping:** -- SQL driver: `TEXT` +- SQL driver: `TEXT` — or `VARCHAR(maxLength)` when a declared index keys the + column, on the same two conditions as the `text` type above. - MongoDB: `String` **UI rendering:** @@ -1095,7 +1122,7 @@ The column each type gets from the SQL driver, per dialect: | ObjectQL Type | PostgreSQL | MySQL | SQLite | |---------------|------------|-------|--------| -| `text` | `TEXT` | `TEXT` | `TEXT` | +| `text` / `textarea` / `html` | `TEXT` \* | `TEXT` \* | `TEXT` \* | | `email` / `url` / `phone` | `VARCHAR(255)` | `VARCHAR(255)` | `VARCHAR(255)` | | `number` / `currency` / `percent` | `REAL` | `FLOAT` | `REAL` | | `date` | `DATE` | `DATE` | `TEXT` (`YYYY-MM-DD`) | @@ -1110,6 +1137,14 @@ The column each type gets from the SQL driver, per dialect: | `formula` | *(no column — virtual)* | *(no column)* | *(no column)* | | `json` / `location` / `address` | `JSON` | `JSON` | `TEXT` (JSON) | +\* The text family is `VARCHAR(maxLength)` rather than `TEXT`, on all three +dialects, when **both** hold: the field declares a `maxLength` of 768 or less, +**and** a declared index keys the column (field-level `unique`, or an entry in +the object's `indexes[]`). A column no index touches stays `TEXT` whatever it +declares, and so does a keyed column whose bound exceeds 768 characters — see +the `text` type above for why, and for what the driver does when a keyed column +cannot be bounded. + Any field flagged `multiple: true` becomes a `JSON` column regardless of its type. Relationship columns are plain id strings with no database `FOREIGN KEY` constraint (see `lookup` above). The MongoDB driver is schemaless — it issues no diff --git a/packages/drivers/driver-sql/src/schema-drift.ts b/packages/drivers/driver-sql/src/schema-drift.ts index 26c28db693..6457cd4eed 100644 --- a/packages/drivers/driver-sql/src/schema-drift.ts +++ b/packages/drivers/driver-sql/src/schema-drift.ts @@ -1050,6 +1050,49 @@ export function expectedIndexes(args: { return out.filter((i) => i.columns.every((c) => physicalColumns.has(c))); } +/** + * Every column any declared index on this object will use as a KEY PART, mapped + * to whether at least one of those indexes is UNIQUE. + * + * Computed from the same two normalizers {@link expectedIndexes} composes — + * field-level `unique` through {@link uniqueIndexesFromFields}, object-level + * `indexes[]` through {@link normalizeDeclaredIndex} — so "which columns end up + * in a key" has ONE answer, shared by the index sync that creates them and by + * the DDL that has to make them keyable in the first place (#11374). + * + * ⚠️ Deliberately NOT filtered by `physicalColumns`, unlike `expectedIndexes`: + * its caller runs BEFORE the columns exist — deciding a column's TYPE is the + * whole reason it asks — so a filter against the physical set would answer + * "nothing is indexed" on exactly the CREATE TABLE path that needs the answer. + * + * The UNIQUE flag is carried because the two dispositions genuinely differ on + * MySQL: a bounded key part is merely a storage choice for an ordinary index, + * but it is the CONSTRAINT itself for a unique one (see + * `mysqlKeyableTextLength` and the refusal it feeds). + */ +export function indexedKeyColumns(args: { + table: string; + fields: Record; + tenantField: string | null; + declaredIndexes?: DeclaredIndexInput[]; +}): Map { + const { table, fields, tenantField, declaredIndexes } = args; + const out = new Map(); + const record = (idx: ExpectedIndex) => { + for (const column of idx.columns) { + const prev = out.get(column); + if (prev) prev.unique ||= idx.unique; + else out.set(column, { unique: idx.unique }); + } + }; + for (const idx of uniqueIndexesFromFields(table, fields, tenantField)) record(idx); + for (const idx of Array.isArray(declaredIndexes) ? declaredIndexes : []) { + const norm = normalizeDeclaredIndex(table, idx, tenantField); + if (norm) record(norm); + } + return out; +} + /** * The two names a tenant-scoped field's *legacy* single-column unique index * could have been materialized under before #3696: diff --git a/packages/drivers/driver-sql/src/sql-driver-keyed-text-mysql.test.ts b/packages/drivers/driver-sql/src/sql-driver-keyed-text-mysql.test.ts new file mode 100644 index 0000000000..d63aa91e57 --- /dev/null +++ b/packages/drivers/driver-sql/src/sql-driver-keyed-text-mysql.test.ts @@ -0,0 +1,289 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #11374 — a text-family field that a declared index KEYS ON. + * + * ## The defect this pins, and why nothing caught it + * + * `createColumn` mapped the whole text family to an unbounded `TEXT`, ignoring + * the field's declared `maxLength`. MySQL refuses a TEXT/BLOB column in a key + * without a prefix length, so the `CREATE TABLE` succeeded and the following + * `ALTER TABLE … ADD [UNIQUE] INDEX` failed — the table landed on disk WITHOUT + * the constraint it declared, and the object stayed registered-but-broken. + * Measured on MySQL 8.0.46 before the fix: **36 of the 44 platform objects** + * failed schema-sync that way, every one with `ER_BLOB_KEY_WITHOUT_LENGTH`, so + * an auth stack could not stand up its own schema on a driver that + * `createDefaultDatasourceDriverFactory` maps `mysql`/`mysql2` onto. + * + * The reason no gate saw it is the reason this file is HERE, in `driver-sql`, + * rather than anywhere nearer the platform objects: the required + * `Temporal Conformance (live PG + MySQL)` job runs `pnpm --filter + * @objectstack/driver-sql test` with `OS_TEST_MYSQL_URL` set, and every suite in + * this package builds its tables with an explicit `knex.string()` (VARCHAR) — + * so the one job with a live MySQL attached never exercised the TEXT mapping at + * all. A new file in this directory needs no CI change to be covered: the + * globalSetup creates its per-file database from the same ledger every other + * live file uses. + * + * ## What each half is worth + * + * The SQLite block runs everywhere, including Test Core, and pins the emission + * rule itself (`varchar(maxLength)` for a keyed bounded field, TEXT otherwise). + * The MySQL block is the one that reds on the pre-fix tree, because SQLite has + * no keyable-type restriction to violate — which is precisely how a defect that + * only MySQL can express stayed invisible. + * + * Opt-in for the live half — needs a real server: + * + * OS_TEST_MYSQL_URL=mysql://root:root@127.0.0.1:3306/conformance \ + * pnpm --filter @objectstack/driver-sql test + */ + +import { describe, it, expect, afterEach } from 'vitest'; +import { SqlDriver } from '../src/index.js'; +import { MYSQL_CELL, dialectCell, declareDialectCell } from './live-dialect-matrix.testkit.js'; + +/** + * One object whose every text field is keyed, unkeyed, bounded or unbounded on + * purpose — the four corners of {@link SqlDriver.keyableTextLength}'s decision + * in one table, so a change to that decision cannot be green in three corners + * and wrong in the fourth. + */ +const BOUNDED_TABLE = 'os11374_bounded'; +const boundedObject = () => ({ + name: BOUNDED_TABLE, + fields: { + // keyed + bounded → varchar(n). The shape `sys_user.phone_number` has. + slug: { type: 'text', maxLength: 64 }, + // keyed + bounded, in a COMPOSITE → same rule, no special case. + locale: { type: 'text', maxLength: 16 }, + // NOT keyed, bounded → stays TEXT. The row-size guard: `varchar(65000)` on + // utf8mb4 is 260000 bytes and exceeds MySQL's 65535-byte row limit, so + // bounding an unkeyed column could make a working table un-creatable. + body: { type: 'text', maxLength: 65000 }, + // NOT keyed, unbounded → stays TEXT (unchanged behaviour). + notes: { type: 'text' }, + }, + indexes: [ + { fields: ['slug'], unique: true }, + { fields: ['slug', 'locale'], unique: false }, + ], +}); + +/** The same object with the keyed column's bound REMOVED — the refusal case. */ +const UNBOUNDED_TABLE = 'os11374_unbounded'; +const unboundedObject = () => ({ + name: UNBOUNDED_TABLE, + fields: { token: { type: 'text' } }, + indexes: [{ fields: ['token'], unique: true }], +}); + +/** + * Bounded, but WIDER than a utf8mb4 key part can be. Measured, not assumed: + * `varchar(768)` takes a unique index and `varchar(769)` is refused with + * `ER_TOO_LONG_KEY: max key length is 3072 bytes`. This is the shape + * `sys_oauth_access_token.token` (`maxLength: 1024`) has, and it must land in + * the SAME refusal as the unbounded case rather than trading one server error + * for a less legible one. + */ +const TOO_WIDE_TABLE = 'os11374_too_wide'; +const tooWideObject = () => ({ + name: TOO_WIDE_TABLE, + fields: { token: { type: 'text', maxLength: 1024 } }, + indexes: [{ fields: ['token'], unique: true }], +}); + +// ── The emission rule, on a dialect every runner has ──────────────────────── + +describe('keyed text columns take their declared maxLength (#11374)', () => { + let driver: SqlDriver; + + afterEach(async () => { + await driver?.disconnect().catch(() => {}); + }); + + it('emits varchar(maxLength) for a keyed bounded field and TEXT for the rest', async () => { + driver = new SqlDriver(dialectCell('sqlite').config()); + await driver.initObjects([boundedObject()]); + const info: Record = + await (driver as any).knex(BOUNDED_TABLE).columnInfo(); + + // SQLite reports the declared type verbatim, which is what we are pinning: + // the DDL the driver ASKED for, independent of any dialect's enforcement. + expect(String(info.slug?.type).toLowerCase()).toContain('varchar'); + expect(String(info.slug?.maxLength ?? '')).toBe('64'); + expect(String(info.locale?.type).toLowerCase()).toContain('varchar'); + expect(String(info.locale?.maxLength ?? '')).toBe('16'); + + // The two unkeyed columns are untouched by the rule — a bound is emitted + // because an INDEX needs it, never merely because a field declared one. + expect(String(info.body?.type).toLowerCase()).toBe('text'); + expect(String(info.notes?.type).toLowerCase()).toBe('text'); + }); + + it('leaves a keyed field TEXT when it declares no usable bound', async () => { + driver = new SqlDriver(dialectCell('sqlite').config()); + await driver.initObjects([unboundedObject(), tooWideObject()]); + const unbounded: any = await (driver as any).knex(UNBOUNDED_TABLE).columnInfo(); + const tooWide: any = await (driver as any).knex(TOO_WIDE_TABLE).columnInfo(); + expect(String(unbounded.token?.type).toLowerCase()).toBe('text'); + // Bounded, but past the key ceiling: a `varchar(1024)` column would only + // swap `ER_BLOB_KEY_WITHOUT_LENGTH` for `ER_TOO_LONG_KEY` on MySQL. + expect(String(tooWide.token?.type).toLowerCase()).toBe('text'); + }); +}); + +// ── The half only a live MySQL can measure ────────────────────────────────── + +declareDialectCell(MYSQL_CELL, 'keyed text columns (#11374)', (cell) => { + describe('keyed text columns on live MySQL (#11374)', () => { + let driver: SqlDriver; + + afterEach(async () => { + for (const t of [BOUNDED_TABLE, UNBOUNDED_TABLE, TOO_WIDE_TABLE, 'os11374_prefix']) { + await driver?.execute(`drop table if exists ${t}`).catch(() => {}); + } + await driver?.disconnect().catch(() => {}); + }); + + it('creates the declared indexes — the whole defect, in one assertion', async () => { + driver = new SqlDriver(cell.config()); + await driver.execute(`drop table if exists ${BOUNDED_TABLE}`).catch(() => {}); + + // Pre-fix this REJECTED with `ER_BLOB_KEY_WITHOUT_LENGTH`, leaving the + // table on disk without either index. + await driver.initObjects([boundedObject()]); + + const types = await columnTypes(driver, BOUNDED_TABLE); + expect(types.slug).toBe('varchar(64)'); + expect(types.locale).toBe('varchar(16)'); + expect(types.body).toBe('text'); + expect(types.notes).toBe('text'); + + // Presence is the only proof: the sync degrades some index failures into + // a log rather than a throw, so "initObjects resolved" is not evidence. + const indexes = await indexNames(driver, BOUNDED_TABLE); + expect(indexes).toContain('uniq_os11374_bounded_slug'); + expect(indexes).toContain('idx_os11374_bounded_slug_locale'); + + // And it is a REAL unique over the whole value, not a prefix: MySQL + // reports `sub_part` on a prefixed key part, and null on a full one. + const [{ SUB_PART: subPart }] = (await rowsOf( + driver, + `select sub_part as SUB_PART from information_schema.statistics + where table_schema = database() and table_name = ? and index_name = ?`, + [BOUNDED_TABLE, 'uniq_os11374_bounded_slug'], + )) as any[]; + expect(subPart ?? null).toBeNull(); + }); + + it('refuses an unkeyable column by name instead of weakening the constraint', async () => { + driver = new SqlDriver(cell.config()); + await driver.execute(`drop table if exists ${UNBOUNDED_TABLE}`).catch(() => {}); + + // Loud, and specifically loud: the raw server error names a column in a + // table that was just created successfully, which reads as an index + // quirk rather than an object whose declared uniqueness is now absent. + await expect(driver.initObjects([unboundedObject()])).rejects.toThrow( + /cannot create index 'uniq_os11374_unbounded_token'.*declares no `maxLength`/s, + ); + + // ⛔ The negative half, and the point of the whole disposition: no index + // was substituted. A prefix index here would have made `initObjects` + // resolve and left a constraint that means something else. + expect(await indexNames(driver, UNBOUNDED_TABLE)).toEqual([]); + }); + + it('gives a bound past the key ceiling the same named refusal', async () => { + driver = new SqlDriver(cell.config()); + await driver.execute(`drop table if exists ${TOO_WIDE_TABLE}`).catch(() => {}); + await expect(driver.initObjects([tooWideObject()])).rejects.toThrow( + /cannot create index 'uniq_os11374_too_wide_token'.*wider than 768 characters/s, + ); + expect(await indexNames(driver, TOO_WIDE_TABLE)).toEqual([]); + }); + + /** + * The measurement that DISQUALIFIED the prefix-index route, kept executable + * so it cannot be re-argued from intuition. + * + * A prefix index is a transparent access-path choice on an ORDINARY index — + * MySQL narrows with it and rechecks the full value. On a UNIQUE index it is + * not: the constraint becomes uniqueness of the PREFIX, which is *stricter* + * than the one the object declared. The failure direction is the surprise — + * not "two duplicates slip through" but "two genuinely different values are + * rejected as duplicates". On `sys_session.token` that is a valid sign-in + * refused with a duplicate-key error. + */ + it('MEASUREMENT: a prefix-unique index rejects two DIFFERENT values sharing the prefix', async () => { + driver = new SqlDriver(cell.config()); + await driver.execute(`drop table if exists os11374_prefix`).catch(() => {}); + await driver.execute( + `create table os11374_prefix ( + id varchar(64) not null primary key, + token text, + unique key uniq_prefix (token(191)) + )`, + ); + + const shared = 'A'.repeat(191); + const first = `${shared}ZZZZ-tenant-alpha`; + const second = `${shared}QQQQ-tenant-beta`; + expect(first).not.toBe(second); + + await driver.execute(`insert into os11374_prefix (id, token) values (?, ?)`, [ + 'r1', + first, + ] as any); + + await expect( + driver.execute(`insert into os11374_prefix (id, token) values (?, ?)`, [ + 'r2', + second, + ] as any), + ).rejects.toThrow(/Duplicate entry/i); + + // One row, from two distinct tokens: the second was lost to a constraint + // the object never declared. + const [{ N: n }] = (await rowsOf( + driver, + `select count(*) as N from os11374_prefix`, + )) as any[]; + expect(Number(n)).toBe(1); + }); + }); +}); + +async function rowsOf(driver: SqlDriver, sql: string, bindings: unknown[] = []): Promise { + const res: any = await driver.execute(sql, bindings as any); + if (Array.isArray(res) && Array.isArray(res[0])) return res[0]; + return Array.isArray(res) ? res : (res?.rows ?? []); +} + +/** `column_type` (with length), lower-cased — `varchar(64)`, `text`. */ +async function columnTypes(driver: SqlDriver, table: string): Promise> { + const rows = await rowsOf( + driver, + `select column_name, column_type from information_schema.columns + where table_schema = database() and table_name = ?`, + [table], + ); + const out: Record = {}; + for (const r of rows as any[]) { + out[String(r.COLUMN_NAME ?? r.column_name)] = String( + r.COLUMN_TYPE ?? r.column_type, + ).toLowerCase(); + } + return out; +} + +/** Non-PRIMARY index names physically present on the table. */ +async function indexNames(driver: SqlDriver, table: string): Promise { + const rows = await rowsOf( + driver, + `select distinct index_name from information_schema.statistics + where table_schema = database() and table_name = ? and index_name <> 'PRIMARY'`, + [table], + ); + return (rows as any[]).map((r) => String(r.INDEX_NAME ?? r.index_name)).sort(); +} diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index c33fb40e48..fa767ee7c8 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -81,6 +81,7 @@ import { expectedIndexes, fieldHasColumn, GLOBAL_TENANT, + indexedKeyColumns, isIndexDriftOp, isUniqueScopeDeclared, legacyUniqueReplacements, @@ -8035,6 +8036,14 @@ export class SqlDriver implements IDataDriver { protected async ensureShardTable(shardName: string, obj: { fields?: Record; tenancy?: any }): Promise { const builtinColumns = new Set(['id', 'created_at', 'updated_at']); const exists = await this.knex.schema.hasTable(shardName); + // #11374: a shard carries the base table's declared indexes (below), so its + // columns need the same keyable-text decision the managed path makes. + const keyedColumns = indexedKeyColumns({ + table: shardName, + fields: obj.fields ?? {}, + tenantField: this.resolveTenantField(shardName), + declaredIndexes: (obj as any).indexes, + }); if (!exists) { await this.knex.schema.createTable(shardName, (table) => { table.string('id').primary(); @@ -8042,7 +8051,7 @@ export class SqlDriver implements IDataDriver { this.createAuditTimestampColumn(table, 'updated_at'); for (const [name, field] of Object.entries(obj.fields ?? {})) { if (builtinColumns.has(name)) continue; - this.createColumn(table, name, field); + this.createColumn(table, name, field, keyedColumns.get(name)); } }); } else { @@ -8051,7 +8060,7 @@ export class SqlDriver implements IDataDriver { await this.knex.schema.alterTable(shardName, (table) => { for (const [name, field] of Object.entries(obj.fields ?? {})) { if (!existingColumns.includes(name)) { - this.createColumn(table, name, field); + this.createColumn(table, name, field, keyedColumns.get(name)); } } }); @@ -8476,6 +8485,18 @@ export class SqlDriver implements IDataDriver { // rejects CREATE TABLE with two columns of the same name). const builtinColumns = new Set(['id', 'created_at', 'updated_at']); + // #11374: which columns this object's indexes will KEY ON, resolved before + // any DDL runs. `createColumn` needs it to decide whether a bounded text + // field takes `varchar(maxLength)` (keyable) or TEXT — a decision that is + // only makeable at CREATE time, since no dialect turns a TEXT column into + // a keyable one afterwards. + const keyedColumns = indexedKeyColumns({ + table: tableName, + fields: obj.fields ?? {}, + tenantField, + declaredIndexes: (obj as any).indexes, + }); + if (!exists) { await this.knex.schema.createTable(tableName, (table) => { table.string('id').primary(); @@ -8484,7 +8505,7 @@ export class SqlDriver implements IDataDriver { if (obj.fields) { for (const [name, field] of Object.entries(obj.fields)) { if (builtinColumns.has(name)) continue; - this.createColumn(table, name, field); + this.createColumn(table, name, field, keyedColumns.get(name)); } } }); @@ -8501,7 +8522,7 @@ export class SqlDriver implements IDataDriver { if (obj.fields) { for (const [name, field] of Object.entries(obj.fields)) { if (!existingColumns.includes(name)) { - this.createColumn(table, name, field); + this.createColumn(table, name, field, keyedColumns.get(name)); } } } @@ -10240,6 +10261,19 @@ export class SqlDriver implements IDataDriver { // `code` / `errno` / `message` / `cause`; see // `@objectstack/types`' `unique-violation.ts` for why it is the one // name for this question. + // #11374: MySQL's refusal of a TEXT key part names a column in a table + // that was just created successfully, which reads as an index quirk + // rather than what it is — the object is now registered with its + // declared uniqueness absent. Re-throw the SAME failure carrying the + // field-level fix; the boot still fails loudly, it just says why. + const unkeyable = await this.explainUnkeyableTextColumn(tableName, name, columns, e); + if (unkeyable) { + (this.logger.error ?? this.logger.warn)(unkeyable, msg); + throw Object.assign(new Error(`${unkeyable} (server said: ${msg})`), { + code: (e as { code?: string }).code, + cause: e, + }); + } if (nullSafe.size > 0 && isUniqueViolationError(e)) { // Existing rows violate the NULL-safe unique — the #5030 defect made // visible. Do not take the boot down: the declared constraint is not @@ -12527,7 +12561,110 @@ export class SqlDriver implements IDataDriver { table.timestamp(name).defaultTo(this.knex.fn.now()); } - protected createColumn(table: Knex.CreateTableBuilder, name: string, field: any) { + /** + * The widest `varchar(n)` one utf8mb4 key part can hold on InnoDB: 3072 bytes + * of index key ÷ 4 bytes per character (#11374). + * + * Measured on MySQL 8.0.46 (DYNAMIC row format, the 8.0 default) rather than + * read off a doc page: `varchar(768) UNIQUE` creates, `varchar(769) UNIQUE` + * is refused with `ER_TOO_LONG_KEY: Specified key was too long; max key + * length is 3072 bytes`. + * + * The bound is per KEY, not per column, so a wide composite can exceed it + * while every part is individually legal — this constant cannot prevent that, + * and does not try to. It is the single-part ceiling below which emitting a + * bounded column is known to help; {@link explainUnkeyableTextColumn} handles + * whatever the server still refuses. + */ + protected static readonly MAX_KEYABLE_VARCHAR_CHARS = 768; + + /** + * The `varchar(n)` a KEYED text-family field should take, or `null` to leave + * the column TEXT (#11374). + * + * `null` has exactly two causes, and both are deliberate non-events rather + * than failures here: + * + * - the field declares no `maxLength` — there is no bound to emit, and + * inventing one would impose a truncation boundary (or, under + * `STRICT_TRANS_TABLES`, an outright write refusal — measured: + * `ER_DATA_TOO_LONG`) that the author never declared; + * - the declared bound is wider than a single key part can be, where a + * `varchar(n)` column would simply trade `ER_BLOB_KEY_WITHOUT_LENGTH` for + * `ER_TOO_LONG_KEY`. + * + * Either way the column stays TEXT and, on MySQL, the index over it is + * refused with the diagnostic {@link explainUnkeyableTextColumn} writes. + */ + protected keyableTextLength(field: any): number | null { + const declared = (field as { maxLength?: unknown }).maxLength; + const n = typeof declared === 'string' ? Number(declared) : declared; + if (typeof n !== 'number' || !Number.isInteger(n) || n <= 0) return null; + if (n > SqlDriver.MAX_KEYABLE_VARCHAR_CHARS) return null; + return n; + } + + /** + * Turn MySQL's `ER_BLOB_KEY_WITHOUT_LENGTH` / `ER_TOO_LONG_KEY` into a message + * that names the columns at fault and the declaration that fixes them + * (#11374). + * + * Worth the extra `columnInfo()` read because it only happens on the failure + * path, and because the raw server error is actively misleading about where + * the defect is: it names a COLUMN in a table the driver just created + * successfully, so the natural reading is "the table is fine, the index is + * odd" — when in truth the object is now registered-but-broken, its declared + * uniqueness silently absent, and the fix lives in the field's metadata + * rather than anywhere near the index. + * + * ⛔ What this deliberately does NOT do is substitute a prefix index + * (`KEY (col(191))`) and carry on. For an ordinary index that would be a + * transparent access-path choice, but for a UNIQUE one it silently REPLACES + * the declared constraint with a different, stricter one: a prefix-unique + * index enforces uniqueness over the PREFIX, so two genuinely different + * values that happen to share it collide. Measured on MySQL 8.0.46 — + * `UNIQUE KEY (token(191))`, then two distinct 200+ character tokens sharing + * their first 191 characters: the second insert was rejected with + * `ER_DUP_ENTRY` even though the tokens differ. On `sys_session.token` that + * is a valid sign-in refused as a duplicate. A refusal the operator can read + * is strictly better than a constraint that quietly means something else. + */ + protected async explainUnkeyableTextColumn( + tableName: string, + indexName: string, + columns: string[], + cause: unknown, + ): Promise { + const code = (cause as { code?: string } | undefined)?.code; + if (code !== 'ER_BLOB_KEY_WITHOUT_LENGTH' && code !== 'ER_TOO_LONG_KEY') return null; + let offenders: string[] = []; + try { + const info: Record = await this.knex(tableName).columnInfo(); + offenders = columns.filter((c) => /text|blob/i.test(String(info?.[c]?.type ?? ''))); + } catch { + // Introspection is a nicety here; the advice below holds without it. + } + const named = + offenders.length > 0 + ? `Column(s) ${offenders.map((c) => `"${c}"`).join(', ')} are stored as TEXT` + : 'One or more of its key columns is stored as TEXT'; + return ( + `[sql-driver] cannot create index '${indexName}' on "${tableName}" — MySQL refuses a TEXT/BLOB ` + + `column in a key without a key length. ${named} because the field declares no \`maxLength\` (or one ` + + `wider than ${SqlDriver.MAX_KEYABLE_VARCHAR_CHARS} characters, the most a utf8mb4 key part can hold). ` + + `Declare \`maxLength\` on the field(s) so the column is emitted as varchar(n) and can be keyed ` + + `(#11374). The table exists but this index does NOT, so any uniqueness it declared is currently ` + + `unenforced. A prefix index is deliberately not substituted: on a UNIQUE index it constrains the ` + + `prefix rather than the value, and rejects two different values that share one.` + ); + } + + protected createColumn( + table: Knex.CreateTableBuilder, + name: string, + field: any, + keyed?: { unique: boolean }, + ) { if (field.multiple) { table.json(name); return; @@ -12546,9 +12683,51 @@ export class SqlDriver implements IDataDriver { case 'text': case 'textarea': case 'html': - case 'markdown': - col = table.text(name); + case 'markdown': { + // #11374: a text-family column that some declared index KEYS ON is + // emitted as `varchar(maxLength)` rather than TEXT, whenever the field + // declared a bound this dialect can key on. + // + // MySQL refuses a TEXT/BLOB column in a key without a prefix length + // (`ER_BLOB_KEY_WITHOUT_LENGTH`), so an unbounded TEXT is not merely a + // slower key there — it is not a key at all: `CREATE TABLE` succeeds and + // the following `ALTER TABLE … ADD [UNIQUE] INDEX` fails, leaving the + // table on disk WITHOUT the constraint it declared. Measured on MySQL + // 8.0.46 before this branch existed: 36 of the 44 platform objects + // failed schema-sync that way, so an auth stack could not stand up its + // own schema on a driver `createDefaultDatasourceDriverFactory` maps + // `mysql`/`mysql2` onto. + // + // The bound is the field's OWN `maxLength` — nothing is invented here. + // `schema-drift.ts` already treats `varchar(field.maxLength)` as the + // expected physical shape of a bounded field (its `widen_varchar` / + // `narrow_varchar` ops say so in as many words); this is the emitter + // finally agreeing with the differ. `Field.string` has always taken + // knex's `varchar(255)`, so a bounded text field is now LESS arbitrary + // than its string sibling, not more. + // + // Applied on every dialect rather than under `isMysql`, deliberately: + // the alternative is one declaration with two enforcement answers, so + // the same app would refuse an over-length write on MySQL and accept it + // on Postgres. Dialect-divergent enforcement of one declared bound is + // the defect class this repo's conformance matrices exist to close, and + // a `varchar(n)` is exactly what the SQLite and Postgres columns would + // have been had the field been declared `Field.string`. + // + // ⚠️ Scope, both halves load-bearing: + // - KEYED only. A non-indexed `Field.text({ maxLength: 65000 })` stays + // TEXT: `varchar(65000)` on utf8mb4 is 260000 bytes and blows MySQL's + // 65535-byte ROW limit, which would turn a working table into an + // un-creatable one. Off-page TEXT has no such cost, and a column no + // index keys on has nothing to gain from a bound. + // - BOUNDED only. `maxLength` absent, or wider than a key part can be + // ({@link keyableTextLength}), leaves the column TEXT — see + // {@link explainUnkeyableTextColumn} for what happens next, which is + // a named refusal and never a silently weaker constraint. + const keyable = keyed ? this.keyableTextLength(field) : null; + col = keyable === null ? table.text(name) : table.string(name, keyable); break; + } case 'integer': case 'int': col = table.integer(name);