Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/driver-sql-keyed-text-maxlength.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
---
"@objectstack/driver-sql": minor
---

**Fix:** a text-family field that a declared index keys on is emitted as `varchar(maxLength)` instead of an unbounded `TEXT`, so the index MySQL previously refused can actually be created (#11374).

`createColumn` mapped the whole text family (`text` / `textarea` / `html` / `markdown`) to an unbounded `TEXT`, ignoring the field's own declared `maxLength`. MySQL refuses a `TEXT`/`BLOB` column in a key without a prefix length, and the two halves of schema-sync fail *separately*: the `CREATE TABLE` succeeds, then `ALTER TABLE … ADD [UNIQUE] INDEX` fails with `ER_BLOB_KEY_WITHOUT_LENGTH`. The table therefore lands on disk **without the constraint it declared**, and the object stays registered-but-broken. Measured on a live MySQL 8.0.46: **36 of the 44 platform objects** failed schema-sync this way, so a stack whose `default` datasource is MySQL could not stand up its own schema — the dev-admin seed never landed and first sign-in returned `401 INVALID_EMAIL_OR_PASSWORD`. Honouring the declared bound takes that to **12**.

**The bound is the field's own `maxLength` — nothing is invented.** `schema-drift.ts` already treated `varchar(field.maxLength)` as the expected physical shape of a bounded field (its `widen_varchar` / `narrow_varchar` ops say so in as many words); this is the emitter finally agreeing with the differ. On MySQL that removes a permanent destructive drift finding: `columnInfo()` reports `maxLength: 65535` for a `TEXT` column, so every bounded text field already reported `narrow_varchar` ("metadata caps at 32 chars but the column allows 65535") against a column the driver itself had created.

**Scope, both halves load-bearing.** The bound is emitted only for a column some declared index **keys on** — a non-indexed `Field.text({ maxLength: 65000 })` stays `TEXT`, because `varchar(65000)` on utf8mb4 is 260000 bytes and would blow MySQL's 65535-byte row limit, turning a working table into an un-creatable one. And only where the bound is **usable as a key part**: `maxLength` absent, or wider than 768 characters (3072 index bytes ÷ 4 bytes per utf8mb4 character — measured: `varchar(768)` takes a unique index, `varchar(769)` is refused with `ER_TOO_LONG_KEY`), leaves the column `TEXT` and the index refused with a message naming the field and the declaration that fixes it.

**⚠️ Graded `minor`, not `patch`: this changes declared behaviour on newly created tables.** A keyed bounded text column now enforces its declared length where the dialect enforces `varchar` (Postgres and MySQL), so a write longer than `maxLength` that previously landed in an unbounded `TEXT` is now refused — under `STRICT_TRANS_TABLES`, with `ER_DATA_TOO_LONG`. That is the declaration becoming enforced rather than a new restriction, and it is exactly what makes the column indexable, but it is a behaviour change and is named here as one. **Existing tables are unaffected**: schema-sync is additive and never rewrites a column that is already present.

**A prefix index is deliberately NOT substituted for an unkeyable column.** For an ordinary index that would be a transparent access-path choice, but for a `UNIQUE` one it silently replaces the declared constraint with a stricter one — uniqueness of the *prefix*. Measured on MySQL 8.0.46 with `UNIQUE KEY (token(191))` and two distinct 200+ character tokens sharing their first 191 characters: the second insert was rejected with `ER_DUP_ENTRY` **even though the tokens differ**. On `sys_session.token` that is a valid sign-in refused as a duplicate. The refusal an operator can read is strictly better than a constraint that quietly means something else.
45 changes: 40 additions & 5 deletions content/docs/protocol/objectql/types.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -77,10 +77,35 @@ company_name:
```

**Database mapping:**
- SQL driver: `TEXT` on every dialect. `maxLength` is enforced by record
validation, not by the column type — the DDL does not read it.
- SQL driver: `TEXT` on every dialect — **except when a declared index keys the
column**, where it is `VARCHAR(maxLength)` instead. Both conditions are
required: the field declares a `maxLength` of **768 or less** (the widest key
part utf8mb4 allows — MySQL's 3072-byte index limit ÷ 4 bytes per character),
**and** some declared index keys on it (field-level `unique`, or an entry in
the object's `indexes[]`). So the `company_name` field above stays `TEXT`
unless an index names it, and a keyed field declaring `maxLength: 1024` stays
`TEXT` too.
- MongoDB: `String`

<Callout type="info">
**Why the bound follows the index.** MySQL refuses a `TEXT`/`BLOB` column in a
key without a prefix length, so an unbounded keyed column makes its own index
un-creatable — the `CREATE TABLE` succeeds and the `ALTER TABLE … ADD INDEX`
fails, leaving the table without the constraint it declared. Bounding an
*unkeyed* column would buy nothing and cost something: `TEXT` is stored
off-page, while a wide `VARCHAR` counts against MySQL's 65535-byte row limit.

When a keyed column cannot be bounded, the driver **refuses the index** and
names the field, rather than silently substituting a prefix index — a
prefix-`UNIQUE` constrains the prefix rather than the value, so it rejects two
different values that happen to share one.

`maxLength` is enforced by record validation on **every** field. On a keyed
column it is *additionally* enforced by the column type, so PostgreSQL and
MySQL refuse an over-length write at the database as well (SQLite does not
enforce `VARCHAR` length).
</Callout>

**UI rendering:**
```html
<input type="text" maxlength="255" required />
Expand All@@ -104,7 +129,8 @@ description:
```

**Database mapping:**
- SQL driver: `TEXT`
- SQL driver: `TEXT` — or `VARCHAR(maxLength)` when a declared index keys the
column, on the same two conditions as the `text` type above.
- MongoDB: `String`

**UI rendering:**
Expand All@@ -129,7 +155,8 @@ bio:
```

**Database mapping:**
- SQL driver: `TEXT`
- SQL driver: `TEXT` — or `VARCHAR(maxLength)` when a declared index keys the
column, on the same two conditions as the `text` type above.
- MongoDB: `String`

**UI rendering:**
Expand DownExpand Up@@ -1095,7 +1122,7 @@ The column each type gets from the SQL driver, per dialect:

| ObjectQL Type | PostgreSQL | MySQL | SQLite |
|---------------|------------|-------|--------|
| `text` | `TEXT` | `TEXT` | `TEXT` |
| `text` / `textarea` / `html` | `TEXT` \* | `TEXT` \* | `TEXT` \* |
| `email` / `url` / `phone` | `VARCHAR(255)` | `VARCHAR(255)` | `VARCHAR(255)` |
| `number` / `currency` / `percent` | `REAL` | `FLOAT` | `REAL` |
| `date` | `DATE` | `DATE` | `TEXT` (`YYYY-MM-DD`) |
Expand All@@ -1110,6 +1137,14 @@ The column each type gets from the SQL driver, per dialect:
| `formula` | *(no column — virtual)* | *(no column)* | *(no column)* |
| `json` / `location` / `address` | `JSON` | `JSON` | `TEXT` (JSON) |

\* The text family is `VARCHAR(maxLength)` rather than `TEXT`, on all three
dialects, when **both** hold: the field declares a `maxLength` of 768 or less,
**and** a declared index keys the column (field-level `unique`, or an entry in
the object's `indexes[]`). A column no index touches stays `TEXT` whatever it
declares, and so does a keyed column whose bound exceeds 768 characters — see
the `text` type above for why, and for what the driver does when a keyed column
cannot be bounded.

Any field flagged `multiple: true` becomes a `JSON` column regardless of its
type. Relationship columns are plain id strings with no database `FOREIGN KEY`
constraint (see `lookup` above). The MongoDB driver is schemaless — it issues no
Expand Down
43 changes: 43 additions & 0 deletions packages/drivers/driver-sql/src/schema-drift.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -1050,6 +1050,49 @@ export function expectedIndexes(args: {
return out.filter((i) => i.columns.every((c) => physicalColumns.has(c)));
}

/**
* Every column any declared index on this object will use as a KEY PART, mapped
* to whether at least one of those indexes is UNIQUE.
*
* Computed from the same two normalizers {@link expectedIndexes} composes —
* field-level `unique` through {@link uniqueIndexesFromFields}, object-level
* `indexes[]` through {@link normalizeDeclaredIndex} — so "which columns end up
* in a key" has ONE answer, shared by the index sync that creates them and by
* the DDL that has to make them keyable in the first place (#11374).
*
* ⚠️ Deliberately NOT filtered by `physicalColumns`, unlike `expectedIndexes`:
* its caller runs BEFORE the columns exist — deciding a column's TYPE is the
* whole reason it asks — so a filter against the physical set would answer
* "nothing is indexed" on exactly the CREATE TABLE path that needs the answer.
*
* The UNIQUE flag is carried because the two dispositions genuinely differ on
* MySQL: a bounded key part is merely a storage choice for an ordinary index,
* but it is the CONSTRAINT itself for a unique one (see
* `mysqlKeyableTextLength` and the refusal it feeds).
*/
export function indexedKeyColumns(args: {
table: string;
fields: Record<string, any>;
tenantField: string | null;
declaredIndexes?: DeclaredIndexInput[];
}): Map<string, { unique: boolean }> {
const { table, fields, tenantField, declaredIndexes } = args;
const out = new Map<string, { unique: boolean }>();
const record = (idx: ExpectedIndex) => {
for (const column of idx.columns) {
const prev = out.get(column);
if (prev) prev.unique ||= idx.unique;
else out.set(column, { unique: idx.unique });
}
};
for (const idx of uniqueIndexesFromFields(table, fields, tenantField)) record(idx);
for (const idx of Array.isArray(declaredIndexes) ? declaredIndexes : []) {
const norm = normalizeDeclaredIndex(table, idx, tenantField);
if (norm) record(norm);
}
return out;
}

/**
* The two names a tenant-scoped field's *legacy* single-column unique index
* could have been materialized under before #3696:
Expand Down
Loading
Loading