diff --git a/.changeset/null-list-comparand-refused.md b/.changeset/null-list-comparand-refused.md new file mode 100644 index 0000000000..6ff2f50e18 --- /dev/null +++ b/.changeset/null-list-comparand-refused.md @@ -0,0 +1,44 @@ +--- +"@objectstack/spec": minor +--- + +feat(spec): refuse `null` in list-comparand positions — `$in` / `$nin` members and `$between` bounds (#13357, #13495) + +**BREAKING** accept-set narrowing on the filter contract, shipped as `minor` +under the repo's launch-window convention for breaking changes. Maintainer +ruling 2026-08-31 (option C): the contract refuses the shape loudly at the +validation entrance, and the cross-backend divergence it used to reach becomes +constructively unreachable — ⛔ no cross-backend alignment (#5299 stays +declined), and the reference matcher's own answers for these shapes are sealed +behind the refusal, not repaired. + +What is refused, and where: + +- **Runtime door** (`assertListComparandShapes`, run inside `parseFilterAST` + and at the engine seam on every verb): a `null` member of `$in` / `$nin`, + and a `null` `$between` bound, are refused with the platform envelope + (`INVALID_FILTER` / 400). Previously the shape reached the backends, where + the SQL family, the mingo path and the reference matcher answered it three + ways — the matcher even disagreed with itself across the two readings of + "no value" (#13357's table). +- **Schema door** (`SetOperatorSchema` / `FieldOperatorsSchema`): a `null` + member is refused at parse time with a pointed message, the same + check-not-type-change mechanism as the #7596 `{ $field }` member refusal. + A `null` `$between` endpoint never parsed (the endpoint union is + `number | Date | string`); it now gets the pointed message instead of zod's + generic union text. + +The refusal text prescribes the ruling's explicit spelling: "one of […] OR has +no value" is `{"$or": [{"$in": […]}, {"$null": true}]}`, and `{"$null": false}` +is the has-a-value half. The carve-out is null-shaped and nothing wider: +`$in: []` / `$nin: []` stay the declared predicates they are, every non-null +member type keeps parsing (#5041's and #5234's member questions stand +untouched), and `$eq: null` is a separate surface (#13494, ruled separable). + +**Migration.** A filter refused by the new checks was already answered +inconsistently across backends, so it had no portable meaning to preserve. +Spell the intent explicitly: `$or: [{$in: […]}, {$null: true}]` for +"one of […] or empty", `{$null: false}` (or `$and` with it) for the +has-a-value direction, and `$gte` / `$lte` for a half-open range. + + diff --git a/content/docs/references/data/filter.mdx b/content/docs/references/data/filter.mdx index 1ebd121234..0de8978293 100644 --- a/content/docs/references/data/filter.mdx +++ b/content/docs/references/data/filter.mdx @@ -123,8 +123,8 @@ Type: `[FilterArray](#filterarray)[]` | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **$in** | `any[]` | optional | Membership list. Members are literal values of any type the column stores. A `{ $field }` reference is NOT a member shape: no backend resolves one inside a list — put it in a scalar comparison ($eq/$ne/$gt/$gte/$lt/$lte) instead. | -| **$nin** | `any[]` | optional | Membership list. Members are literal values of any type the column stores. A `{ $field }` reference is NOT a member shape: no backend resolves one inside a list — put it in a scalar comparison ($eq/$ne/$gt/$gte/$lt/$lte) instead. | +| **$in** | `any[]` | optional | Membership list. Members are literal values of any type the column stores. A `{ $field }` reference is NOT a member shape: no backend resolves one inside a list — put it in a scalar comparison ($eq/$ne/$gt/$gte/$lt/$lte) instead. null is NOT a member shape either: state absence explicitly with the null predicate — "one of […] OR has no value" is `{ "$or": [{ "$in": […] }, { "$null": true }] }`. | +| **$nin** | `any[]` | optional | Membership list. Members are literal values of any type the column stores. A `{ $field }` reference is NOT a member shape: no backend resolves one inside a list — put it in a scalar comparison ($eq/$ne/$gt/$gte/$lt/$lte) instead. null is NOT a member shape either: state absence explicitly with the null predicate — "one of […] OR has no value" is `{ "$or": [{ "$in": […] }, { "$null": true }] }`. | --- diff --git a/packages/drivers/driver-memory/src/memory-null-list-member-unreachable.test.ts b/packages/drivers/driver-memory/src/memory-null-list-member-unreachable.test.ts new file mode 100644 index 0000000000..bf02306aa4 --- /dev/null +++ b/packages/drivers/driver-memory/src/memory-null-list-member-unreachable.test.ts @@ -0,0 +1,112 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#13357] Ruling point 3's NEGATIVE pin, matcher side: a refused null list + * member cannot reach this package's reference matcher. + * + * # What was ruled (2026-08-31, option C) + * + * #13357 measured that the reference matcher answers `$in: [null]` / + * `$nin: [null]` differently across the two readings of "no value" (a stored + * `null` vs an absent key) while `$null` / `$ne: null` agree — and that the + * SQL family answers the same filters a third way. The ruling REFUSES the + * shape at the contract's validation entrance (`@objectstack/spec`, + * `assertListComparandShapes`, run inside `parseFilterAST` and at the engine + * seam) instead of aligning the backends: the divergence becomes + * constructively unreachable, ⛔ deliberately not repaired (「⛔ 不单独修一个 + * 到不了的路径」), so NOTHING in this file asserts what the matcher would + * have answered. `memory-matcher-null-value-and-comparand.test.ts` keeps + * those arms deliberately absent for the same reason. + * + * # What this file pins, and its honest boundary + * + * A direct caller of this driver — this repo's own conformance suites, an + * embedder — compiles its filter with `parseFilterAST` and hands the result + * over (`filter-comparand-shape.ts`'s #9228 section is the ruling that put + * the gate on that face for exactly this caller). This file drives that + * pipeline end to end and pins that for every refused shape it ABORTS at the + * compile face, on BOTH fixture readings, before any row is consulted: the + * evaluation step is provably never reached because the compile step throws. + * The engine half (every verb, driver-call witness) is pinned in + * `@objectstack/objectql`'s `engine-filter-array-lowering.test.ts`; the + * wire/protocol face runs the same `parseFilterAST`. + * + * The boundary, stated rather than hidden: `match()` and + * `InMemoryDriver.find()` remain plain library functions — a caller that + * skips the compile face meets only this package's own + * `assertFilterConditionShape`, which is deliberately NOT extended to the + * null-member rule (⛔ 不做跨后端对齐工程). That boundary is the same one + * every #5869 refusal has had since #9228, and it is not widened here. + */ + +import { describe, it, expect } from 'vitest'; +import { parseFilterAST } from '@objectstack/spec/data'; + +import { match } from './memory-matcher.js'; + +type Refusal = Error & { code?: string; status?: number }; + +/** The card's own fixture, in both readings of "no value" (#13357). */ +const NULLED_ROWS: Array> = [ + { id: '1', name: 'a' }, + { id: '3', name: null }, +]; +const MISSING_ROWS: Array> = [ + { id: '1', name: 'a' }, + { id: '3' }, +]; + +/** + * The direct-caller pipeline, exactly as the module note describes it: compile + * first, evaluate second. The refusal has to land in step one — if compile + * returns, the matcher HAS been reached and the pin below fails on the + * sentinel rather than on a missing throw. + */ +function compileThenMatch(rows: Array>, where: unknown): string[] { + const condition = parseFilterAST(where); + return rows.filter((row) => match(row, condition)).map((row) => String(row.id)); +} + +const refusalOf = (run: () => unknown): Refusal => { + try { + run(); + } catch (e) { + return e as Refusal; + } + throw new Error('expected the compile face to refuse this filter, but it returned'); +}; + +describe('[#13357] a refused null list member cannot reach the matcher (ruled 2026-08-31)', () => { + it.each([ + ['$in: [null]', { name: { $in: [null] } }], + ['$nin: [null]', { name: { $nin: [null] } }], + ['$between: [null, null]', { name: { $between: [null, null] } }], + ['$between: [null, max]', { name: { $between: [null, 'z'] } }], + ['$between: [min, null]', { name: { $between: ['a', null] } }], + ])('%s aborts at the compile face on BOTH readings of "no value"', (_label, where) => { + // Record-independent by construction — the compile face never sees a row — + // so the two readings that split the matcher (#13357's table) cannot even + // be posed. Driving both anyway is the point of the pin: neither fixture + // gets an answer, so there is no divergence left to observe. + for (const rows of [NULLED_ROWS, MISSING_ROWS]) { + const err = refusalOf(() => compileThenMatch(rows, where)); + expect(err.code, _label).toBe('INVALID_FILTER'); + expect(err.status, _label).toBe(400); + } + }); + + it('the pipeline itself is real — a legal list compiles and the matcher answers', () => { + // Positive control: without it, the refusals above would also "pass" if + // compileThenMatch were broken outright. + expect(compileThenMatch(NULLED_ROWS, { name: { $in: ['a'] } })).toEqual(['1']); + expect(compileThenMatch(MISSING_ROWS, { name: { $nin: ['a'] } })).toEqual(['3']); + }); + + it('an EMPTY list still passes the same face — the refusal is null-shaped, not list-shaped', () => { + // `$in: []` / `$nin: []` are declared predicates ("matches nothing" / + // "matches everything") and PR #13630 pins them downstream; the carve-out + // must not catch them. + expect(compileThenMatch(NULLED_ROWS, { name: { $in: [] } })).toEqual([]); + expect(compileThenMatch(NULLED_ROWS, { name: { $nin: [] } })).toEqual(['1', '3']); + }); +}); diff --git a/packages/objectql/src/engine-filter-array-lowering.test.ts b/packages/objectql/src/engine-filter-array-lowering.test.ts index 0ee8a9355f..1b4124f207 100644 --- a/packages/objectql/src/engine-filter-array-lowering.test.ts +++ b/packages/objectql/src/engine-filter-array-lowering.test.ts @@ -483,6 +483,69 @@ describe('Door 2 lowers FilterArray to FilterCondition before the driver (#5158) expect(err.message).toMatch(/where\.amount\.\$between/); }); + // ── [#13357] the null carve-out, ruled 2026-08-31: refused at this seam, ── + // ── so the drivers' three-way null-member divergence is UNREACHABLE ────── + // + // Ruling point 3's negative pin, engine half: the witness is the recording + // driver's call log, not the thrown envelope alone — an envelope asserted + // without the log would also pass if a DRIVER had thrown it, i.e. if the + // refusal had not preceded the dispatch, which is the one thing these + // assertions exist to prove. The compile-face half (`parseFilterAST`, both + // input forms) is pinned in `@objectstack/spec`'s + // `filter-comparand-shape.test.ts`; the matcher-side statement lives in + // driver-memory's `memory-null-list-member-unreachable.test.ts`. ⛔ Nothing + // here asserts what any backend WOULD have answered — the divergence is + // sealed, not reconciled (#5299 stays declined). + + it.each([ + ['$in: [null]', { stage: { $in: [null] } }], + ['$nin: [null]', { stage: { $nin: [null] } }], + ['$between: [null, null]', { amount: { $between: [null, null] } }], + ['$between: [null, max]', { amount: { $between: [null, 20] } }], + ])('a null list member is refused on EVERY verb before any driver call — %s', async (_l, where) => { + await expect(engine.find('deal', { where })) + .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' }); + await expect(engine.findOne('deal', { where })) + .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' }); + await expect(engine.count('deal', { where } as unknown as EngineCountOptions)) + .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' }); + await expect(engine.aggregate('deal', { + where: where as unknown as EngineAggregateOptions['where'], + groupBy: ['stage'], + aggregations: [{ function: 'count', field: 'id', alias: 'n' }], + })).rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' }); + await expect(engine.update('deal', { amount: 1 }, { where, multi: true } as any)) + .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' }); + await expect(engine.delete('deal', { where, multi: true } as any)) + .rejects.toMatchObject({ status: 400, code: 'INVALID_FILTER' }); + // The negative half: refused BEFORE the store — no read, no write, no row + // moved. (The count() control below adds its own read, so it runs after.) + expect(reads).toHaveLength(0); + expect(writes).toHaveLength(0); + expect(await engine.count('deal')).toBe(3); + }); + + it('the null-member refusal is not vacuous — the same list WITHOUT null reaches the driver', async () => { + // Positive control for the zero-call reading above: one member removed, + // same operator, same field, and the dispatch happens. + const rows = await engine.find('deal', { where: { stage: { $in: ['won'] } } }); + expect(reads).toHaveLength(1); + expect(lastWhere()).toEqual({ stage: { $in: ['won'] } }); + expect(rows.map((r: any) => r.id).sort()).toEqual(['d1', 'd3']); + }); + + it('a nested null member is refused at its own path, engine prefix and all', async () => { + const err = await engine.find( + 'deal', + { where: { $or: [{ stage: { $nin: [null] } }] } }, + ).then(() => null, (e: any) => e); + expect(err?.status).toBe(400); + expect(err?.code).toBe('INVALID_FILTER'); + expect(err.message).toMatch(/^find\('deal'\): /); + expect(err.message).toContain('where.$or[0].stage.$nin[0]'); + expect(reads).toHaveLength(0); + }); + // ── what must KEEP working: the declared list shapes ─────────────────── it('a proper list comparand still reaches the driver untouched', async () => { diff --git a/packages/spec/src/data/filter-comparand-shape.test.ts b/packages/spec/src/data/filter-comparand-shape.test.ts index f0d9da1051..28b4c4665d 100644 --- a/packages/spec/src/data/filter-comparand-shape.test.ts +++ b/packages/spec/src/data/filter-comparand-shape.test.ts @@ -98,6 +98,68 @@ describe('the list-comparand shape door (#5869) runs inside parseFilterAST (#922 expect(err.status).toBe(400); }); + // ── the null carve-out, ruled 2026-08-31 (#13357; $between is #13495) ── + + it.each([ + ['$in, lowered array form', [['stage', 'in', [null]]]], + ['$in, object passthrough', { stage: { $in: [null] } }], + ['$nin, lowered array form', [['stage', 'not_in', [null]]]], + ['$nin, object passthrough', { stage: { $nin: [null] } }], + ['$in with a real neighbour', { stage: { $in: ['won', null] } }], + ])('refuses a null list MEMBER — %s', (_label, where) => { + const err = refusalOf(() => parseFilterAST(where)); + expect(err.code).toBe(StandardErrorCode.enum.INVALID_FILTER); + expect(err.status).toBe(400); + }); + + it.each([ + ['[null, null]', { at: { $between: [null, null] } }], + ['[null, max]', { at: { $between: [null, '2026-07-15'] } }], + ['[min, null]', { at: { $between: ['2026-07-01', null] } }], + ])('refuses a null $between BOUND — %s', (_label, where) => { + const err = refusalOf(() => parseFilterAST(where)); + expect(err.code).toBe(StandardErrorCode.enum.INVALID_FILTER); + expect(err.status).toBe(400); + }); + + it('the null-member refusal prescribes the ruling\'s explicit spelling', () => { + // 2026-08-31: 「等于 X 或为空」的合法拼法是显式的 $or + $null — the + // refusal must spell it out, in both halves, and still name operator, + // field, position and authoring spellings (the #5346/#5348 contract). + const err = refusalOf(() => parseFilterAST({ stage: { $nin: [null] } })); + expect(err.message).toMatch(/^Operator "\$nin" on field "stage"/); + expect(err.message).toContain('where.stage.$nin[0]'); + expect(err.message).toContain('{"$or": [{"stage": {"$in": […]}}, {"stage": {"$null": true}}]}'); + expect(err.message).toContain('{"$null": false}'); + expect(err.message).toMatch(/Authoring spellings: nin, not_in, notin/); + expect(err.message).toMatch(/UNFILTERED result set/); + }); + + it('the null-bound refusal points at the offending index and the working alternatives', () => { + const err = refusalOf(() => parseFilterAST({ at: { $between: ['2026-07-01', null] } })); + expect(err.message).toMatch(/^Operator "\$between" on field "at" requires two non-null bounds/); + expect(err.message).toContain('where.at.$between[1]'); + expect(err.message).toContain('"$gte"/"$lte"'); + expect(err.message).toContain('{"at": {"$null": true}}'); + expect(err.message).toMatch(/UNFILTERED result set/); + }); + + it('a null member is refused at its own path inside $and / $or / $not too', () => { + expect(refusalOf(() => parseFilterAST({ $not: { stage: { $in: [null] } } })).message) + .toContain('where.$not.stage.$in[0]'); + expect(refusalOf(() => parseFilterAST({ $or: [{ stage: { $nin: [null] } }] })).message) + .toContain('where.$or[0].stage.$nin[0]'); + }); + + it('refuses ONLY null — falsy and empty-ish members are values, not absence', () => { + // The carve-out is null-shaped and nothing wider: #5041's and #5234's + // member questions stand untouched, and every falsy VALUE keeps working. + expect(parseFilterAST({ n: { $in: [0, false, ''] } })).toEqual({ n: { $in: [0, false, ''] } }); + expect(parseFilterAST({ n: { $nin: [0, false, ''] } })).toEqual({ n: { $nin: [0, false, ''] } }); + expect(parseFilterAST({ at: { $between: ['', ''] } })).toEqual({ at: { $between: ['', ''] } }); + expect(parseFilterAST({ n: { $between: [0, 0] } })).toEqual({ n: { $between: [0, 0] } }); + }); + // ── the wording contract (#5346 / #5348), unchanged by the move ──────── it('names the operator, the field, what arrived, where, and the fix', () => { @@ -134,6 +196,13 @@ describe('the list-comparand shape door (#5869) runs inside parseFilterAST (#922 [['stage', 'not_in', 'won']], [['stage', 'in', 'won']], [['amount', 'between', 5]], + // The 2026-08-31 null carve-out (#13357/#13495): the prescribed $or + + // $null spelling makes these the LONGEST messages this door assembles, + // so they live inside the same unrelaxed bound. + { stage: { $in: [null] } }, + { stage: { $nin: [null] } }, + { close_date: { $between: [null, null] } }, + { close_date: { $between: ['2026-07-01', null] } }, ]) { const err = refusalOf(() => parseFilterAST(where, "find('deal')")); expect(err.message.length, JSON.stringify(where)).toBeLessThan(500); diff --git a/packages/spec/src/data/filter-comparand-shape.ts b/packages/spec/src/data/filter-comparand-shape.ts index 65ada8d0e5..beeac28599 100644 --- a/packages/spec/src/data/filter-comparand-shape.ts +++ b/packages/spec/src/data/filter-comparand-shape.ts @@ -80,13 +80,30 @@ * predicate — "matches nothing" and "matches everything" respectively — and * both drivers say so in as many words. Arity is not this gate's business; * only "is it a list at all". - * - **The MEMBER types of any list.** `$between`'s members are checked by - * nobody (`driver-sql` checks arity and nothing else, and #5041 measured the + * - **The MEMBER types of any list — except `null`, refused BY RULING (next + * section).** `$between`'s members are otherwise checked by nobody + * (`driver-sql` checks arity and nothing else, and #5041 measured the * member case and deliberately left it — ISO date strings are a legitimate * range on every backend); `$in`/`$nin` members are #5234's subject, on the * `driver-sql` object-syntax face, and are not re-judged here. The six * accepted comparand TYPES are a different question, answered one file over * by {@link normalizeFilterComparandTypes} (#7872). + * + * ## Refused BY RULING, 2026-08-31: a `null` list member (#13357) + * + * The carve-out is null-shaped and nothing wider. A `null` member of `$in` / + * `$nin`, and a `null` `$between` endpoint (#13495's shape), are refused at + * this door: no two backend camps ever agreed on what a null in a + * list-comparand position matches (`$in: [null]` / `$nin: [null]` split the + * reference matcher's two readings of "no value" while `$null` / `$ne: null` + * agree, and the SQL family's `NOT IN` answer is unconditional), and the + * maintainer ruled the divergence constructively unreachable rather than + * reconciled — ⛔ no cross-backend alignment; #5299 stays declined, and the + * matcher's own answers for these shapes are sealed behind this refusal, not + * repaired. "Equals X or has no value" has an explicit spelling — + * `$or: [{$in: […]}, {$null: true}]` — and the refusal text prescribes it. + * #5041's question (ISO date strings as legitimate `$between` bounds) and + * #5234's (object members on the `driver-sql` face) stand untouched. * - **A field spec with no `$` keys** (`{ author: { name: 'x' } }`) — a * deep-equality comparand to `driver-memory` and `driver-mongodb` alike. This * gate does not descend into one: a comparand is data, and a stricter reading @@ -251,6 +268,68 @@ function malformedRangeComparandError( ); } +/** + * A `null` MEMBER of a membership list — refused BY RULING, 2026-08-31 + * (#13357); see the module note's "Refused BY RULING" section. + * + * The prescription is the ruling's own sentence: absence is stated with the + * null predicate, explicitly — `$or: [{$in: […]}, {$null: true}]` — never + * smuggled into a membership list no two backends read alike. The `$nin` + * author's usual intent ("has a value and it is not one of […]") is the + * `{$null: false}` half, so both halves are named. Same #5346/#5348 wording + * contract as {@link nonListComparandError}: operator, field, position, + * corrected shape, front-loaded; the schema door's twin is + * `nullListComparandMemberMessage` (`./filter.zod.ts`), reconciled by pin. + */ +function nullListMemberError( + context: string | undefined, + op: string, + field: string, + index: number, + path: string, +): Error { + const spellings = LIST_COMPARAND_OPERATORS.get(op) ?? []; + return invalidFilterComparandError( + context, + `Operator "${op}" on field "${field}" does not accept null as a list member ` + + `(at ${path}[${index}]). No two backends agree on what it matches; state absence ` + + `explicitly with the null predicate: {"$or": [{"${field}": {"$in": […]}}, ` + + `{"${field}": {"$null": true}}]} is "one of […] OR has no value"; {"$null": false} ` + + `is "has a value". Authoring spellings: ${spellings.join(', ')}. The filter was NOT ` + + `applied, and an unapplied filter would have returned the UNFILTERED result set.`, + ); +} + +/** + * A `$between` bound that is `null` — the same 2026-08-31 ruling, #13495's + * shape, folded into this door. + * + * Its own message rather than an extension of + * {@link malformedRangeComparandError}: that one's leading sentence is kept + * verbatim from `driver-sql`'s and `driver-memory`'s ARITY arms (#5240's + * one-condition-one-wording rule), and a null bound is a different condition + * that NO driver refuses — there is no existing wording to share, and welding + * the new condition onto the arity sentence would misdescribe a well-arity + * `[null, max]` as "not a [min, max] value array". + */ +function nullRangeBoundError( + context: string | undefined, + field: string, + value: unknown, + index: number, + path: string, +): Error { + return invalidFilterComparandError( + context, + `Operator "$between" on field "${field}" requires two non-null bounds. Received null ` + + `at ${path}[${index}] of ${shapePreview(value)}. A half-open range is "$gte"/"$lte"; ` + + `"in range OR has no value" is {"$or": [{"${field}": {"$between": [min, max]}}, ` + + `{"${field}": {"$null": true}}]}. The authoring spelling that lowers to "$between" is ` + + `"between". The filter was NOT applied, and an unapplied filter would have returned ` + + `the UNFILTERED result set.`, + ); +} + /** * Walk one `FilterCondition` and refuse every list-shaped operator whose * comparand cannot be one. @@ -327,10 +406,26 @@ function assertFieldListComparands( if (!Array.isArray(comparand) || comparand.length !== 2) { throw malformedRangeComparandError(context, field, comparand, `${path}.${op}`); } + // Shape first (a list at all), then the null carve-out (2026-08-31 + // ruling, #13495's shape) — so `$between: null` keeps the arity message + // it has always had and only a well-arity pair can reach this check. + const nullBound = comparand.indexOf(null); + if (nullBound !== -1) { + throw nullRangeBoundError(context, field, comparand, nullBound, `${path}.${op}`); + } continue; } if (!Array.isArray(comparand)) { throw nonListComparandError(context, op, field, comparand, `${path}.${op}`); } + // The null-member carve-out (2026-08-31 ruling, #13357). `indexOf` is the + // O(list) pass the door already pays for `$between`'s arity — strict + // equality, so `undefined`, `''`, `0` and `false` members are untouched, + // and an EMPTY list never enters the branch (it has no members; `$in: []` + // / `$nin: []` stay the declared predicates they are). + const nullMember = comparand.indexOf(null); + if (nullMember !== -1) { + throw nullListMemberError(context, op, field, nullMember, `${path}.${op}`); + } } } diff --git a/packages/spec/src/data/filter.test.ts b/packages/spec/src/data/filter.test.ts index 275b4fffe7..cd072f8b8c 100644 --- a/packages/spec/src/data/filter.test.ts +++ b/packages/spec/src/data/filter.test.ts @@ -215,14 +215,41 @@ describe('SetOperatorSchema', () => { }); it('leaves every other member shape open — the list is field-AGNOSTIC', () => { - // The check removes ONE shape. A plain object member is still a legal - // membership value (a JSON column stores documents), and narrowing the - // member type would refuse working filters this schema cannot judge. - expect(SetOperatorSchema.safeParse({ $in: [{ nested: 1 }, null, 3, new Date()] }).success) + // The checks remove TWO shapes — `{ $field }` (#7596) and, since the + // 2026-08-31 ruling (#13357), `null` — and nothing else. A plain object + // member is still a legal membership value (a JSON column stores + // documents), and narrowing the member TYPE would refuse working filters + // this schema cannot judge. `null` sat in this openness fixture until + // that ruling carved it out; its refusal is pinned by name below. + expect(SetOperatorSchema.safeParse({ $in: [{ nested: 1 }, 3, new Date()] }).success) .toBe(true); expect(SetOperatorSchema.safeParse({ $in: [] }).success).toBe(true); }); + it('refuses a null member of $in / $nin — ruled 2026-08-31 (#13357)', () => { + for (const op of ['$in', '$nin'] as const) { + const result = SetOperatorSchema.safeParse({ [op]: ['won', null] }); + expect(result.success, op).toBe(false); + const issue = result.error?.issues[0]; + expect(issue?.path).toEqual([op, 1]); + expect(issue?.message).toContain(`${op} member at index 1`); + // The prescription is the ruling's own explicit spelling — both + // halves, so the `$nin` author finds the has-a-value direction too. + expect(issue?.message).toContain('{"$or": [{"$in": […]}, {"$null": true}]}'); + expect(issue?.message).toContain('{"$null": false}'); + } + }); + + it('refuses a null $between endpoint with the pointed message, not zod\'s generic union text', () => { + // `null` never passed the endpoint union (it is none of number | Date | + // string) — the ruling adds the POINTED message for it (#13495's shape). + const result = RangeOperatorSchema.safeParse({ $between: [null, '2026-01-01'] }); + expect(result.success).toBe(false); + const messages = (result.error?.issues ?? []).map((i) => i.message).join('\n'); + expect(messages).toContain('$between endpoint at index 0'); + expect(messages).toContain('{"$null": true}'); + }); + it('is matched by the enforced copy — FieldOperatorsSchema', () => { expect(FieldOperatorsSchema.safeParse({ $in: [{ $field: 'budget' }] }).success).toBe(false); expect(FieldOperatorsSchema.safeParse({ $nin: [{ $field: 'budget' }] }).success).toBe(false); diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 07a64dfdb0..1f71470c2d 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -301,21 +301,56 @@ function listPositionFieldReferenceMessage(position: string): string { } /** - * [#7596] `$in` / `$nin`, with the `{ $field }` member ruled out by name. + * [#13357] The author-facing refusal for a `null` in a LIST-comparand position + * — every `$in` / `$nin` member, and both `$between` endpoints (#13495's + * shape). Ruled 2026-08-31; the same one-builder-for-four-positions shape as + * {@link listPositionFieldReferenceMessage}, because it is likewise one ruling. + * + * The prescription sentence is the ruling's own: "equals X or has no value" + * has an explicit spelling, `$or: [{$in: […]}, {$null: true}]`, and the + * refusal names it rather than leaving the author to guess. The runtime twin + * is `nullListMemberError` / `nullRangeBoundError` + * (`./filter-comparand-shape.ts`), reconciled by pin — two moments, one + * author, no contradiction. + */ +function nullListComparandMemberMessage(position: string): string { + return ( + `null is not a valid ${position}. No two backends agree on what a null in a ` + + 'list-comparand position matches (the SQL family answers a NULL under NOT IN ' + + 'unconditionally; the JS matchers split over the two readings of "no value"). State ' + + 'absence explicitly with the null predicate instead: ' + + '{"$or": [{"$in": […]}, {"$null": true}]} is "one of […] OR has no value", and ' + + '{"$null": false} is the has-a-value half. ' + + 'Ruled 2026-08-31: a null list member is refused at the validation entrance.' + ); +} + +/** + * [#7596] `$in` / `$nin`, with the `{ $field }` member ruled out by name — + * and, since the 2026-08-31 ruling (#13357), the `null` member likewise. * * The members stay `z.any()`: a set-membership list is genuinely heterogeneous * (a `lookup` id, an ISO day, a number), and this schema is field-AGNOSTIC — it * never sees which column the list applies to, so narrowing the member type * would refuse working filters, exactly the finding `RangeOperatorSchema`'s * "why a BARE string" section records for the sibling slot. What IS removable - * is the one shape no backend implements, so it is removed as a check rather - * than as a type change: everything else keeps parsing, `{ $field }` is refused - * with the message it needs, and the generated JSON Schema still describes the - * list as the open one it is. + * is a shape no backend implements — `{ $field }`, which nothing resolves in a + * list, and `null`, which no two backends read alike — so both are removed as + * checks rather than as a type change: everything else keeps parsing, each + * refused shape gets the message it needs, and the generated JSON Schema still + * describes the list as the open one it is. */ const setMembershipSchema = (op: '$in' | '$nin') => z.array(z.any()).superRefine((members, ctx) => { members.forEach((member, index) => { + if (member === null) { + ctx.addIssue({ + code: 'custom', + path: [index], + message: nullListComparandMemberMessage(`${op} member at index ${index}`), + }); + return; + } if (!isFieldReferenceShape(member)) return; ctx.addIssue({ code: 'custom', @@ -325,11 +360,13 @@ const setMembershipSchema = (op: '$in' | '$nin') => }); }); -/** The `describe()` both `$in` and `$nin` carry, stating the one ruled-out member shape. */ +/** The `describe()` both `$in` and `$nin` carry, stating the two ruled-out member shapes. */ const SET_MEMBER_DESCRIPTION = 'Membership list. Members are literal values of any type the column stores. A ' + '{ $field } reference is NOT a member shape: no backend resolves one inside a list' - + ' — put it in a scalar comparison ($eq/$ne/$gt/$gte/$lt/$lte) instead.'; + + ' — put it in a scalar comparison ($eq/$ne/$gt/$gte/$lt/$lte) instead. null is NOT ' + + 'a member shape either: state absence explicitly with the null predicate — ' + + '"one of […] OR has no value" is { "$or": [{ "$in": […] }, { "$null": true }] }.'; /** * Set operators for membership checks. @@ -482,9 +519,15 @@ const RANGE_ENDPOINT_DESCRIPTION = const rangeEndpointSchema = (index: 0 | 1) => z.union([z.number(), z.date(), z.string()], { error: (issue) => - isFieldReferenceShape(issue.input) - ? listPositionFieldReferenceMessage(`$between endpoint at index ${index}`) - : undefined, + // [#13357] `null` never passed this union (it is none of the three + // endpoint types) — what the 2026-08-31 ruling adds here is the POINTED + // message in place of zod's generic union text, the same replace-only + // mechanism the `{ $field }` shape uses one line down. + issue.input === null + ? nullListComparandMemberMessage(`$between endpoint at index ${index}`) + : isFieldReferenceShape(issue.input) + ? listPositionFieldReferenceMessage(`$between endpoint at index ${index}`) + : undefined, }); export const RangeOperatorSchema = lazySchema(() => z.object({