Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix"); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms"); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links"); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default"); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers"); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + '
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode"); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })();
Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/generate-migration-multiple-json-column.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
---
"@objectstack/cli": patch
---

`os generate migration` now gives a `multiple: true` field a JSON column, in both formats. One authored field used to produce two incompatible answers from one config in one run: `Field.lookup({ reference: 'account', multiple: true })` emitted `account?: string[]` from `os generate types` and a scalar `VARCHAR(36)` / `table.uuid('account')` column from the two migration generators, because `multiple` appeared exactly four times in `generate.ts` and all four were on the TypeScript side — `fieldTypeToSql` did not even take the parameter. Nothing warned: the scaffold looks right, the generated TypeScript IS right, and only the column is wrong, so the first symptom was a write of an array into a scalar column. That is the `#field-zoo` failure one layer out — there the DDL switch and `isJsonField` had drifted into two lists inside the driver; here the platform and the *generated* DDL were the two lists.

The authority is the driver's, and it is the flag alone. `SqlDriver.createColumn` short-circuits on `field.multiple` **above** its own `switch (type)`; `isJsonField` is `JSON_COLUMN_TYPES.has(type) || !!field.multiple`; and `fieldHasColumn` opens with `if (field?.multiple) return true` under the comment "Mirrors `SqlDriver.createColumn` exactly … including `multiple` (a JSON column)". Three statements of one rule: a flagged field is a JSON column whatever its element type would have been. Both generators now answer it the same way and in the same place — before the type is consulted at all.

Deliberately **not** the spec's `isMultiValueField`. That predicate is the ADR-0104 D1 *value* contract ("is the persisted value an array") and gates on `MULTI_CAPABLE_TYPES`, so asking it here would answer `VARCHAR` for a `text` field flagged `multiple: true` while the driver gives that same field a JSON column — the identical drift one notch narrower. `FieldSchema` does not refuse the combination either (`multiple` is a plain `z.boolean()` on every field; only `radio` + `multiple` is refused by name), and the generators sit downstream of validation. The two questions have two different owners: the value shape is the spec's, the column is the driver's.

Nothing about the existing per-type vocabularies changes. The scalar answers — including the five that are separately disputed — are byte-for-byte what they were, and a new pin asserts that as a scope fence rather than leaving it to a reading of the diff. `generate-multiple-json-column.pin.test.ts` drives all three generators on one config and pins the agreement across every member of the spec's `MULTI_CAPABLE_TYPES` plus a type outside it, so the type-blindness of the rule is an assertion rather than a comment; it also reads the driver's two statements of the rule, so moving them there fails here.
265 changes: 265 additions & 0 deletions packages/cli/src/commands/generate-multiple-json-column.pin.test.ts
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,265 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* THE #14829 PIN: one authored `multiple: true` field, three surfaces, ONE answer.
*
* ## The defect
*
* `multiple` appeared exactly FOUR times in `generate.ts`, and all four were on
* the TypeScript side (measured at `origin/main` 5bc2f2727a:
* `git grep -n multiple origin/main -- packages/cli/src/commands/generate.ts`):
*
* :562 function fieldTypeToTs(fieldType: string, multiple?: boolean)
* :564 return multiple ? `${base}[]` : base;
* :607 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate types
* :831 const tsType = fieldTypeToTs(fType, !!fieldDef.multiple); // os generate client
*
* Neither migration generator read it, and `fieldTypeToSql` did not even take
* the parameter. So ONE authored field produced two incompatible answers from
* one config in one run — `Field.lookup({ reference: 'account', multiple: true })`
* emitted `account?: string[]` from `os generate types` and a scalar
* `VARCHAR(36)` / `table.uuid('account')` column from the two migration
* generators. Nothing warns: the scaffold looks right, the generated
* TypeScript IS right, and only the column is wrong, so the first symptom is a
* write. That is the `#field-zoo` failure one layer out — there the DDL switch
* and `isJsonField` had drifted into two lists inside the driver; here the
* platform and the GENERATED DDL are the two lists.
*
* ## Which surface is authoritative, and why it is NOT `isMultiValueField`
*
* Measured on `origin/main`, the platform answers "which column does this field
* get" from the FLAG ALONE, before it looks at the type, and says so in three
* places:
*
* packages/drivers/driver-sql/src/sql-driver.ts `createColumn`
* `if (field.multiple) { this.jsonColumn(table, name); return; }` — stated
* above the `switch (type)`, so the element type never gets a vote.
* packages/drivers/driver-sql/src/sql-driver.ts `isJsonField`
* `JSON_COLUMN_TYPES.has(type) || !!field.multiple`
* packages/drivers/driver-sql/src/schema-drift.ts `fieldHasColumn`
* `if (field?.multiple) return true;` — under the comment "Mirrors
* `SqlDriver.createColumn` exactly … everything else — including `multiple`
* (a JSON column) — gets one."
*
* The spec's `isMultiValueField` is a DIFFERENT question with a different
* answer: it is the ADR-0104 D1 VALUE contract ("is the persisted value an
* array"), and it gates on `MULTI_CAPABLE_TYPES` —
* `MULTI_OPTION_TYPES.has(type) || (MULTI_CAPABLE_TYPES.has(type) && multiple)`.
* A generator that asked it instead would answer VARCHAR for a `text` field
* flagged `multiple: true` while the driver gives that same field a JSON
* column — reintroducing this very drift one notch narrower. `FieldSchema`
* does not refuse the combination either (`multiple` is a plain
* `z.boolean().default(false)` on every field; only `radio` + `multiple` is
* refused, by name, in `field.zod.ts`'s superRefine), and the CLI generators
* sit DOWNSTREAM of validation and explicitly serve the unvalidated authoring
* door. So the column authority is the driver's flag-first rule, and this pin
* asserts against that.
*
* `MULTI_CAPABLE_TYPES` is still imported here rather than transcribed — it is
* the roster this pin SWEEPS, so a type added to that spec class is measured on
* the day it lands. It is not the implementation's gate, and the type-blindness
* control below is what states the difference as an assertion.
*
* ## Anti-vacuity
*
* Every arm has a control, because a pin that measured nothing would pass
* loudest of all. The controls are separate `it` blocks with `control —` in
* their names, so a red run says in its own title whether the discriminating
* arm fired or merely the harness: the roster really loaded, the generators
* really emitted, and — the one that matters — the SAME type WITHOUT the flag
* still gets its scalar column, so "JSONB everywhere" cannot pass this file.
*/

import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

import { MULTI_CAPABLE_TYPES } from '@objectstack/spec/data';
import { describe, expect, it } from 'vitest';

import {
generateMigrationSql,
generateMigrationTs,
generateTypesFromConfig,
} from './generate.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const DRIVER_SQL_SRC = path.resolve(HERE, '../../../drivers/driver-sql/src');

/**
* The types swept for the flag. The spec's multi-capable roster (imported, not
* restated) plus `text` — a type that is NOT in that roster and whose scalar
* answer is a varchar, which is what makes the type-blindness of the rule
* assertable rather than merely described.
*/
const FLAGGED_TYPES: readonly string[] = [...MULTI_CAPABLE_TYPES, 'text'];

/** One object carrying, for each swept type, a flagged field and its scalar twin. */
function probeConfig(): Record<string, unknown> {
const fields: Record<string, Record<string, unknown>> = {};
for (const type of FLAGGED_TYPES) {
fields[`multi_${type}`] = { type, multiple: true };
fields[`single_${type}`] = { type };
}
return { objects: { probe: { name: 'probe', label: 'Probe', fields } } };
}

const TYPES_OUT = generateTypesFromConfig(probeConfig());
const SQL_OUT = generateMigrationSql(probeConfig());
const TS_OUT = generateMigrationTs(probeConfig());

/** The `"name" TYPE` column body one field contributes to the SQL migration. */
function sqlColumn(field: string): string {
const m = SQL_OUT.match(new RegExp(`^ {2}"${field}" (.+?),?$`, 'm'));
if (!m) throw new Error(`no SQL column emitted for ${field}`);
return m[1];
}

/** The `table.x('name')…` call one field contributes to the TS migration. */
function tsColumn(field: string): string {
const m = TS_OUT.match(new RegExp(`^ {4}(table\\.\\w+\\('${field}'\\)).*$`, 'm'));
if (!m) throw new Error(`no TS migration column emitted for ${field}`);
return m[1];
}

/** The declared property type one field contributes to the generated interface. */
function tsInterfaceType(field: string): string {
const m = TYPES_OUT.match(new RegExp(`^ {2}${field}\\??: (.+);$`, 'm'));
if (!m) throw new Error(`no interface member emitted for ${field}`);
return m[1];
}

describe('#14829 — `multiple: true` is one answer across all three surfaces', () => {
it('control — the spec multi-capable roster really loaded', () => {
expect(MULTI_CAPABLE_TYPES.size).toBeGreaterThanOrEqual(6);
for (const known of ['select', 'lookup', 'user', 'file', 'image']) {
expect(MULTI_CAPABLE_TYPES.has(known)).toBe(true);
}
// `text` is the type-blindness probe: it must NOT be in the roster, or the
// control below stops distinguishing the flag rule from the value rule.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
});

it('control — all three generators really emitted a table for the probe', () => {
expect(TYPES_OUT).toContain('export interface ProbeRecord {');
expect(SQL_OUT).toContain('CREATE TABLE IF NOT EXISTS "probe" (');
expect(TS_OUT).toContain("await db.schema.createTable('probe'");
// Non-vacuity for the readers: every swept field really reached the output.
expect(FLAGGED_TYPES.length).toBeGreaterThanOrEqual(7);
for (const type of FLAGGED_TYPES) {
expect(() => sqlColumn(`multi_${type}`)).not.toThrow();
expect(() => tsColumn(`multi_${type}`)).not.toThrow();
expect(() => tsInterfaceType(`multi_${type}`)).not.toThrow();
}
});

it('control — the SAME type without the flag still gets its scalar column', () => {
// THE discriminating control. If this file could be satisfied by emitting a
// JSON column for everything, the arms below would prove nothing.
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");
expect(sqlColumn('single_text')).toBe('VARCHAR(255)');
expect(tsColumn('single_text')).toBe("table.string('single_text')");
expect(tsInterfaceType('single_lookup')).toBe('string');
});

for (const type of FLAGGED_TYPES) {
it(`${type} + multiple:true — array TS type AND a JSON column in both migrations`, () => {
const declared = tsInterfaceType(`multi_${type}`);
expect(declared, `os generate types must give a flagged ${type} an array type`)
.toMatch(/\[\]$/);

expect(
sqlColumn(`multi_${type}`),
`os generate migration --format sql gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON (driver-sql createColumn decides `multiple` before ' +
'the type switch), and os generate types called it an array',
).toBe('JSONB');

expect(
tsColumn(`multi_${type}`),
`os generate migration (typescript) gave a flagged ${type} a scalar column while ` +
'the platform stores it as JSON, and os generate types called it an array',
).toBe(`table.jsonb('multi_${type}')`);
});
}

it('the flag decides before the type — a type outside MULTI_CAPABLE_TYPES too', () => {
// Stated as its own assertion because it is the one place this pin departs
// from the spec's value predicate on purpose. `text` is not multi-capable
// under `isMultiValueField`, and the driver gives it a JSON column anyway.
expect(MULTI_CAPABLE_TYPES.has('text')).toBe(false);
expect(sqlColumn('multi_text')).toBe('JSONB');
expect(tsColumn('multi_text')).toBe("table.jsonb('multi_text')");
});

it('nullability still comes from `required`, not from the flag', () => {
const out = generateMigrationSql({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(out).toContain('"tags_req" JSONB NOT NULL');
const ts = generateMigrationTs({
objects: { probe: { name: 'probe', fields: { tags_req: { type: 'lookup', multiple: true, required: true } } } },
});
expect(ts).toContain("table.jsonb('tags_req').notNullable();");
});

// ── The authority, read where it lives ──────────────────────────────────
//
// Source-read rather than imported: `createColumn` is `protected` and needs a
// knex table builder, so driving it would mean a live driver and a built
// `dist`. What has to be pinned is the SHAPE of its decision — flag first,
// type second — and that is legible in the source. If the driver ever moves
// this rule, these fail and whoever moved it re-derives the generators.

it('driver-sql `createColumn` still decides `multiple` BEFORE the type switch', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'sql-driver.ts'), 'utf8');
// Non-vacuity: the file was really read, and the two landmarks really found.
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('protected createColumn(');
expect(start, 'createColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
const switchAt = source.indexOf('switch (type)', start);
expect(switchAt, 'the per-type switch in createColumn moved').toBeGreaterThan(start);

const preSwitch = source.slice(start, switchAt);
expect(
preSwitch,
'driver-sql no longer short-circuits on `field.multiple` before its per-type switch. ' +
'That short-circuit is the authority this pin and the CLI migration generators mirror ' +
'(#14829) — re-derive both sides before changing it.',
).toMatch(/if \(field\.multiple\)/);
expect(preSwitch).toMatch(/this\.jsonColumn\(/);
});

it('driver-sql `fieldHasColumn` still answers the flag before the type', () => {
const source = fs.readFileSync(path.join(DRIVER_SQL_SRC, 'schema-drift.ts'), 'utf8');
expect(source.length).toBeGreaterThan(10_000);
const start = source.indexOf('export function fieldHasColumn(');
expect(start, 'fieldHasColumn moved or was renamed in driver-sql').toBeGreaterThan(0);
expect(source.slice(start, start + 300)).toMatch(/if \(field\?\.multiple\) return true;/);
});

// ── SCOPE FENCE for #14828 — NOT an endorsement ─────────────────────────
//
// These five scalar answers disagree with what the platform stores and were
// left byte-for-byte on purpose (correcting them changes DDL already-generated
// apps have RUN). #14828 owns them. They are asserted here so that changing
// one is a deliberate edit to this block rather than a side effect of a card
// about the `multiple` flag — #14828 must update it when it corrects them.
it('#14828 fence — the five disputed SCALAR answers are untouched by this card', () => {
expect(sqlColumn('single_lookup')).toBe('VARCHAR(36)');
expect(tsColumn('single_lookup')).toBe("table.uuid('single_lookup')");

const other = generateMigrationSql({
objects: { probe: { name: 'probe', fields: {
a: { type: 'autonumber' }, f: { type: 'formula' },
m: { type: 'multiselect' }, v: { type: 'vector' }, d: { type: 'master_detail' },
} } },
});
expect(other).toContain('"a" SERIAL');
expect(other).toContain('"f" TEXT');
expect(other).toContain('"m" TEXT');
expect(other).toContain('"v" VECTOR');
expect(other).toContain('"d" VARCHAR(36)');
});
});
Loading
Loading