diff --git a/.changeset/mongodb-contains-case-sensitive.md b/.changeset/mongodb-contains-case-sensitive.md new file mode 100644 index 0000000000..83c86f8b5b --- /dev/null +++ b/.changeset/mongodb-contains-case-sensitive.md @@ -0,0 +1,40 @@ +--- +"@objectstack/driver-mongodb": patch +--- + +fix(driver-mongodb): the `$contains` family is case-SENSITIVE — the hardcoded `$options: 'i'` is gone (#6682) + +**Row-set change for anyone filtering text on MongoDB.** `$contains`, +`$notContains`, `$startsWith` and `$endsWith` translated to a `$regex` with a +hardcoded `$options: 'i'` beside it, which is MongoDB's full-Unicode case fold. +That flag is off all four arms. `{ name: { $contains: 'acme' } }` no longer +returns `ACME Corp`. + +This is #4706 **Q2 = A** — the family is case-sensitive on every backend — +arriving at the last driver that was on the wrong side of it. #6518 flipped the +SQL family (`GLOB` on the SQLite dialects, `LIKE` over a binary cast on MySQL, +`LIKE` unchanged on Postgres); `formula`, ObjectQL's `having` and +service-analytics' compilers were already case-exact. + +**Both directions of the defect mattered.** The fold OVER-matched — it returned +rows the filter excludes, which on an RLS read scope is over-reach rather than a +loose filter (#3948) — and it folded the whole Unicode range, overshooting the +ASCII-only boundary Q1 = A holds `$icontains` to. + +**If you were relying on the fold, write `$icontains`.** It is the deliberate +case-insensitive spelling, implemented on this driver since #6520, and it folds +ASCII only (`café` does not match `CAFÉ`) — the one domain every backend can +deliver. + +Unchanged: `escapeRegex`, so the comparand is still matched LITERALLY (`a.b` +matches `a.b`, not `axb`), and `$icontains`, whose fold has always lived in the +pattern rather than in `$options`. Every face of this driver — +`find`/`count`/`update`/`delete` and the aggregation `$match` — routes through +the one `translateFilter`, so there is no second answer to align. + +The driver's `FILTER_TEXT_CASES` conformance cell is now CLEARED: the new +server-free suite `mongodb-filter-text-conformance.test.ts` imports the shared +case-set and drives all seventeen cases, rejection rows included, and the DEBT +row for this cell is deleted from `scripts/check-driver-conformance.mjs`. +`driver-memory`'s half of #6682 stays open under the #5499 freeze, so that card +remains open. diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx index e1a112dff7..2446e7131d 100644 --- a/content/docs/protocol/objectql/query-syntax.mdx +++ b/content/docs/protocol/objectql/query-syntax.mdx @@ -302,12 +302,13 @@ metacharacters — `{ name: { $icontains: 'a.b' } }` matches `a.b` and not `axb` One half of the case rules above is still landing: `$contains` / `$startsWith` / `$endsWith` / `$notContains` are case-**sensitive** by ruling and - are so on the SQL family, but the in-memory driver's query path and MongoDB still - fold them over the whole Unicode range — - [#6682](https://github.com/objectstack-ai/objectstack/issues/6682). Until that - lands, prefer `$icontains` when you *want* a fold rather than relying on - `$contains` being loose on those two backends. The shared standard both halves - are measured against is `FILTER_TEXT_CASES` (`@objectstack/spec/data`). + are so on the SQL family and on MongoDB — + [#6682](https://github.com/objectstack-ai/objectstack/issues/6682) removed the + hardcoded `$options: 'i'` that had folded them there. The in-memory driver's + query and analytics faces still fold over the whole Unicode range. Until that + half lands, prefer `$icontains` when you *want* a fold rather than relying on + `$contains` being loose on that backend. The shared standard both halves are + measured against is `FILTER_TEXT_CASES` (`@objectstack/spec/data`). ### `$regex` — removed @@ -920,13 +921,15 @@ a user typing `acme` does not find `ACME Corp`. Only `select` / `status` option *labels* are matched case-insensitively by the expansion itself. - **Measured today, and it does not match that rule yet.** The `$contains` alignment - is [#5702](https://github.com/objectstack-ai/objectstack/issues/5702), so until it - lands the answer is still the driver's: `SqlDriver` compiles a parameterised - `LIKE '%…%'` and the dialect decides (SQLite folds ASCII, Postgres does not), - `driver-mongodb` folds the full Unicode range through a hardcoded `$options: 'i'`, - and `driver-memory`'s query path matches with a case-insensitive regex. Which - driver you run therefore still changes which rows a search returns. Whether the + **Measured today: one driver still does not match that rule.** The `$contains` + alignment landed in two steps — + [#6518](https://github.com/objectstack-ai/objectstack/issues/6518) made `SqlDriver` + case-exact per dialect (`GLOB` on the SQLite dialects, `LIKE` unchanged on + Postgres, `LIKE` over a binary cast on MySQL), and + [#6682](https://github.com/objectstack-ai/objectstack/issues/6682) removed + `driver-mongodb`'s hardcoded `$options: 'i'`. `driver-memory`'s query path still + matches with a case-insensitive regex, so running your tests on the in-memory + double can still return rows a SQL or MongoDB deployment would not. Whether the expansion should emit `$icontains` instead of `$contains` — i.e. whether search is case-insensitive by definition — is a separate question that rides with that issue, because it can only be answered once both operators mean one thing everywhere. diff --git a/packages/drivers/driver-mongodb/src/mongodb-filter-text-conformance.test.ts b/packages/drivers/driver-mongodb/src/mongodb-filter-text-conformance.test.ts new file mode 100644 index 0000000000..0511e0fd7c --- /dev/null +++ b/packages/drivers/driver-mongodb/src/mongodb-filter-text-conformance.test.ts @@ -0,0 +1,219 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#6682] `driver-mongodb` held to `FILTER_TEXT_CASES` — the text-operator + * standard, answered without a server. + * + * ## Why this file only exists now + * + * The cell this file clears carried a measured DEBT row in + * `scripts/check-driver-conformance.mjs` for as long as either half of the + * #4706 ruling was open here, because that gate judges coverage by IMPORT: a + * driver that imports the marker claims the WHOLE case-set, so a face answering + * one requirement and not the other must not import it. The two halves closed + * in different PRs: + * + * | requirement | where it landed | + * |:--|:--| + * | 1 — `$icontains`, ASCII-only (#4706 Q1 = A) | #6520, as a sanctioned one-off inside the #5499 freeze | + * | 2 — the `$contains` family is case-SENSITIVE (#4706 Q2 = A) | #6682, this change | + * | 3 — `$regex` / `$options` REFUSED, naming `$icontains` | #5702 | + * + * `mongodb-icontains.test.ts` drove `FILTER_TEXT_ROWS` and spelled its own + * `$icontains` cases rather than naming the marker, precisely so requirement 1 + * would not flip this cell to covered while requirement 2 was open. With + * requirement 2 answered, the marker comes in here and the ledger row goes in + * the same PR — deleting it without this file fails CONSUMED, keeping it + * alongside this file fails RECONCILED. + * + * ## Why the assertions run in-process rather than against mongod + * + * This package's real-mongod suites are OPT-IN + * (`OS_TEST_MONGODB_MEMORY_SERVER_ENABLED=1`, #5517 — the binary is a ~123 MB + * download that is not reachable on a restricted network), so a suite that + * needed a server would not run in CI, and a standard that does not run is the + * shape #4363 wrote the gate over. `translateFilter` is a pure function and the + * whole of this driver's answer to these cases: the case sensitivity of a + * `$regex` predicate is decided entirely by whether `$options: 'i'` is set + * beside it. So the emitted documents are EVALUATED here, by + * {@link selectIds}, the same judgement `mongodb-filter-logic-translation.test.ts` + * makes for the logic case-set and `mongodb-aggregation-translation.test.ts` + * for the aggregation one. The real-mongod half stays absent and is recorded as + * such on #6682 rather than implied here. + * + * @see FILTER_TEXT_CASES — the standard + * @see https://github.com/objectstack-ai/objectstack/issues/4706 (the ruling) + * @see https://github.com/objectstack-ai/objectstack/issues/6682 (requirement 2 on this driver) + * @see https://github.com/objectstack-ai/objectstack/issues/6518 (the same flip on the SQL family) + */ + +import { describe, it, expect } from 'vitest'; +import { + FILTER_TEXT_CASES, + FILTER_TEXT_ROWS, + type FilterTextCase, + type FilterTextRejectionCase, + type FilterTextRow, +} from '@objectstack/spec/data'; +import { translateFilter } from './mongodb-filter.js'; + +// ── A deliberately strict reader of the emitted document ──────────────────── +// +// Same discipline as `mongodb-filter-logic-translation.test.ts`'s `matchDoc`: +// every shape it does not model is a thrown error, never a silently-true +// predicate. A stand-in more permissive than the real engine turns this suite +// into a green light for broken code. Its own discrimination is proved at the +// bottom of this file — a folding document must FAIL the case-sensitive rows. + +/** Thrown for any shape this matcher does not model — never swallowed. */ +class UnsupportedShape extends Error {} + +/** + * `$options` is modelled rather than ignored, and that is the whole point: the + * defect #6682 fixed was a hardcoded `$options: 'i'`, so a matcher that dropped + * the flag would have reported the fixed answer from the broken translator. + */ +function matchRegexOps(value: unknown, ops: Record): boolean { + const source = ops.$regex; + if (typeof source !== 'string') throw new UnsupportedShape('$regex without a string pattern'); + const options = ops.$options; + if (options !== undefined && typeof options !== 'string') { + throw new UnsupportedShape('$options without a string'); + } + for (const key of Object.keys(ops)) { + if (key !== '$regex' && key !== '$options') { + throw new UnsupportedShape(`unsupported key beside $regex: '${key}'`); + } + } + if (typeof value !== 'string') return false; + return new RegExp(source, options ?? '').test(value); +} + +/** Operators applied to one field's value. */ +function matchOps(value: unknown, ops: Record): boolean { + const keys = Object.keys(ops); + if (keys.includes('$regex') || keys.includes('$options')) return matchRegexOps(value, ops); + for (const [op, arg] of Object.entries(ops)) { + switch (op) { + case '$eq': + if (value !== arg) return false; + break; + case '$ne': + if (value === arg) return false; + break; + case '$not': { + // The per-field negation — the only `$not` MongoDB accepts, and the + // shape `$notContains` leaves here as. + if (!arg || typeof arg !== 'object' || Array.isArray(arg)) { + throw new UnsupportedShape('per-field $not takes an operator document'); + } + if (matchOps(value, arg as Record)) return false; + break; + } + default: + throw new UnsupportedShape(`unsupported field operator '${op}'`); + } + } + return true; +} + +/** Evaluate an emitted MongoDB query document against one fixture row. */ +function matchDoc(row: FilterTextRow, doc: Record): boolean { + for (const [key, value] of Object.entries(doc)) { + if (key.startsWith('$')) throw new UnsupportedShape(`unsupported document operator '${key}'`); + const field = (row as unknown as Record)[key]; + if (value !== null && typeof value === 'object' && !Array.isArray(value)) { + if (!matchOps(field, value as Record)) return false; + } else if (field !== value) { + return false; + } + } + return true; +} + +/** Ids the emitted document selects from the shared fixture, ascending. */ +function selectIds(doc: Record): string[] { + return FILTER_TEXT_ROWS.filter((row) => matchDoc(row, doc)) + .map((row) => row.id) + .sort((a, b) => a.localeCompare(b)); +} + +function isRejection(c: FilterTextCase): c is FilterTextRejectionCase { + return c.expectRejection === true; +} + +const ROWS_CASES = FILTER_TEXT_CASES.filter( + (c): c is Exclude => !isRejection(c), +); +const REJECTION_CASES = FILTER_TEXT_CASES.filter(isRejection); + +// ── The shared standard: the evaluated rows ───────────────────────────────── + +describe('[#6682] translateFilter answers FILTER_TEXT_CASES — the evaluated rows', () => { + for (const testCase of ROWS_CASES) { + it(testCase.name, () => { + const doc = translateFilter(testCase.filter) as Record; + expect(selectIds(doc), `${testCase.note ?? ''}\nemitted: ${JSON.stringify(doc)}`).toEqual([ + ...testCase.expected, + ]); + }); + } +}); + +// ── The shared standard: the refusals ─────────────────────────────────────── + +describe('[#6682] translateFilter answers FILTER_TEXT_CASES — every rejection row', () => { + for (const testCase of REJECTION_CASES) { + it(`${testCase.name} — refused in the ADR-0112 envelope`, () => { + let err: (Error & { code?: string; status?: number }) | undefined; + try { + translateFilter(testCase.filter); + } catch (e) { + err = e as Error & { code?: string; status?: number }; + } + // Not `expected: []`. `FilterTextRejectionCase` is a separate discriminant + // because "translated to a predicate that matched nothing" and "refused to + // run" must be told apart — and `code`/`status` rather than a bare + // `toThrow()` because this driver is the reason that bar exists: its + // `default:` arm threw a bare `new Error` with no envelope until #5702. + expect(err, testCase.note ?? 'expected a refusal').toBeInstanceOf(Error); + expect(err!.code).toBe(testCase.code); + expect(err!.status).toBe(400); + for (const mention of testCase.mustMention) expect(err!.message).toContain(mention); + }); + } + + it('drives every rejection row the table declares', () => { + // A guard on the filter above: if a rejection row joins the table, this + // list moves and the enrolment is re-read rather than silently skipping it. + expect(REJECTION_CASES.map((c) => c.name)).toEqual([ + '$regex is REFUSED, and the refusal names $icontains', + '$regex with $options is REFUSED as one mistake, not two', + 'a dangling $options with no $regex is REFUSED', + 'an empty $icontains comparand is REFUSED', + 'a non-string $icontains comparand is REFUSED', + ]); + }); +}); + +// ── The matcher is not the thing being tested, so it gets tested ──────────── + +describe('[#6682] the in-process matcher discriminates', () => { + it('a folding document FAILS the case-sensitive rows — the #6682 defect itself', () => { + // This is the exact document `translateFilter` emitted before this change. + // If the sweep above can pass with this, it is proving nothing. + expect(selectIds({ name: { $regex: 'acme', $options: 'i' } })).toEqual(['1', '2']); + expect(selectIds({ name: { $regex: 'acme' } })).toEqual(['2']); + }); + + it('models the per-field $not rather than treating it as true', () => { + expect(selectIds({ name: { $not: { $regex: 'acme' } } })).toEqual([ + '1', '3', '4', '5', '6', '7', '8', '9', + ]); + }); + + it('refuses any operator it does not model, rather than answering true', () => { + expect(() => selectIds({ name: { $unmodelled: 1 } })).toThrow(/unsupported field operator/); + expect(() => selectIds({ $where: 'true' })).toThrow(/unsupported document operator/); + }); +}); diff --git a/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts b/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts index 6fb1b6d085..1f97428d9c 100644 --- a/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts +++ b/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts @@ -85,30 +85,61 @@ describe('MongoDB Filter Translator', () => { }); }); + /** + * [#6682] Every pin in here used to carry `$options: 'i'` beside its pattern. + * That flag was MongoDB's full-Unicode fold, and #4706 Q2 = A rules this + * family case-SENSITIVE on every backend — so the assertions are not + * "`$options` deleted", they are the ROW SET the flag was widening: the pins + * below assert the emitted pattern, and `mongodb-filter-text-conformance. + * test.ts` asserts which of `FILTER_TEXT_ROWS` it then selects (`$contains: + * 'acme'` no longer returns `ACME Corp`). + * + * `escapeRegex` is untouched and still pinned here: the comparand is LITERAL + * whichever way case is answered, and conflating the two is how a "case fix" + * turns `a.b` back into a metacharacter. + */ describe('string operators', () => { - it('translates $contains to $regex', () => { + it('translates $contains to a case-SENSITIVE $regex — no $options (#6682)', () => { const result = translateFilter({ name: { $contains: 'test' } }); - expect(result).toEqual({ name: { $regex: 'test', $options: 'i' } }); + expect(result).toEqual({ name: { $regex: 'test' } }); }); - it('translates $startsWith to ^prefix regex', () => { + it('translates $startsWith to a case-SENSITIVE ^prefix regex (#6682)', () => { const result = translateFilter({ name: { $startsWith: 'Pre' } }); - expect(result).toEqual({ name: { $regex: '^Pre', $options: 'i' } }); + expect(result).toEqual({ name: { $regex: '^Pre' } }); }); - it('translates $endsWith to suffix$ regex', () => { + it('translates $endsWith to a case-SENSITIVE suffix$ regex (#6682)', () => { const result = translateFilter({ email: { $endsWith: '.com' } }); - expect(result).toEqual({ email: { $regex: '\\.com$', $options: 'i' } }); + expect(result).toEqual({ email: { $regex: '\\.com$' } }); }); it('escapes special regex characters', () => { const result = translateFilter({ name: { $contains: 'a.b+c' } }); - expect(result).toEqual({ name: { $regex: 'a\\.b\\+c', $options: 'i' } }); + expect(result).toEqual({ name: { $regex: 'a\\.b\\+c' } }); }); - it('translates $notContains', () => { + it('translates $notContains — the negation is case-SENSITIVE too (#6682)', () => { const result = translateFilter({ name: { $notContains: 'spam' } }); - expect(result).toEqual({ name: { $not: { $regex: 'spam', $options: 'i' } } }); + expect(result).toEqual({ name: { $not: { $regex: 'spam' } } }); + }); + + /** + * The flag is gone from the whole family, asserted as an absence rather + * than arm by arm: `$options` is a RETIRED operator (#5702), so a future + * arm reintroducing it would be emitting a spelling this same driver + * refuses on input. + */ + it('emits no $options anywhere in the family (#6682)', () => { + for (const filter of [ + { name: { $contains: 'acme' } }, + { name: { $startsWith: 'acme' } }, + { name: { $endsWith: 'acme' } }, + { name: { $notContains: 'acme' } }, + { name: { $icontains: 'acme' } }, + ] as const) { + expect(JSON.stringify(translateFilter(filter))).not.toContain('$options'); + } }); }); @@ -198,7 +229,9 @@ describe('MongoDB Filter Translator', () => { ['<', ['age', '<', 65], { age: { $lt: 65 } }], ['<=', ['score', '<=', 100], { score: { $lte: 100 } }], ['in', ['status', 'in', ['active', 'pending']], { status: { $in: ['active', 'pending'] } }], - ['contains', ['name', 'contains', 'test'], { name: { $regex: 'test', $options: 'i' } }], + // [#6682] Case-SENSITIVE, like the object spelling it lowers to — the + // authoring sugar must not be a second answer to #4706 Q2. + ['contains', ['name', 'contains', 'test'], { name: { $regex: 'test' } }], ['implicit AND list', [['name', '=', 'Alice'], ['age', '>', 18]], { $and: [{ name: 'Alice' }, { age: { $gt: 18 } }] }], // PREFIX is the declared spelling of a logical join. The infix form this diff --git a/packages/drivers/driver-mongodb/src/mongodb-filter.ts b/packages/drivers/driver-mongodb/src/mongodb-filter.ts index 1cc04b6c2c..9f065be8a9 100644 --- a/packages/drivers/driver-mongodb/src/mongodb-filter.ts +++ b/packages/drivers/driver-mongodb/src/mongodb-filter.ts @@ -743,38 +743,59 @@ function translateFieldOperators( } // String operators → $regex + // + // [#6682] Case-SENSITIVE, and the sensitivity is the absence of one key: + // every arm here used to set `$options: 'i'` beside its pattern, which is + // MongoDB's full-Unicode fold. #4706 Q2 = A rules this family + // case-sensitive on EVERY backend — #6518 flipped the SQL family (GLOB on + // the SQLite dialects, `LIKE` over a binary cast on MySQL), `formula`, + // objectql's `having` and service-analytics were already case-exact, and + // this driver was the last face still folding. The fold both OVER-matched + // (`{ name: { $contains: 'acme' } }` returned `ACME Corp` — rows the + // filter excludes, which on an RLS read scope is over-reach rather than a + // loose filter, #3948) and overshot the ASCII-only boundary Q1 = A holds + // `$icontains` to, since `i` folds `É` to `é` as well. + // + // `escapeRegex` is what keeps the comparand LITERAL, and it is unchanged: + // dropping the flag changes which CASES match, never which characters are + // metacharacters. The deliberate case-insensitive spelling is + // `$icontains` below — one operator, one answer, per #5374. case '$contains': result.$regex = escapeRegex(String(value)); - result.$options = 'i'; break; case '$notContains': - result.$not = { $regex: escapeRegex(String(value)), $options: 'i' }; + // The negated twin needs the same treatment in this ONE place: the + // pattern under `$not` is the same predicate, so a flag left here would + // have excluded rows the positive form includes — the negation widening + // rather than mirroring. + result.$not = { $regex: escapeRegex(String(value)) }; break; case '$startsWith': result.$regex = `^${escapeRegex(String(value))}`; - result.$options = 'i'; break; case '$endsWith': result.$regex = `${escapeRegex(String(value))}$`; - result.$options = 'i'; break; // [#6520] `$icontains` — case-insensitive over ASCII and nothing else. // - // The one arm in this family that does NOT set `$options: 'i'`, and the - // omission is the whole implementation. Mongo's `i` flag folds the full - // Unicode range, so it would match `CAFÉ` against `café` — the answer - // SQLite cannot give and therefore the one the protocol forbids (#4706 - // Q1 = A). The fold lives in the pattern instead, one `[Aa]` class per - // ASCII letter, from the spec's shared `asciiCaseInsensitiveRegexSource` - // — the same source `driver-memory`'s mingo path binds. + // No arm in this family sets `$options: 'i'` any more, and for this one + // the omission is the whole implementation. Mongo's `i` flag folds the + // full Unicode range, so it would match `CAFÉ` against `café` — the + // answer SQLite cannot give and therefore the one the protocol forbids + // (#4706 Q1 = A). The fold lives in the pattern instead, one `[Aa]` class + // per ASCII letter, from the spec's shared + // `asciiCaseInsensitiveRegexSource` — the same source `driver-memory`'s + // mingo path binds. // - // Its four neighbours above ARE `$options: 'i'`, and that is not a - // precedent to copy: it is them folding Unicode for the case-SENSITIVE - // `$contains` family, the open defect #6682 tracks on this driver. + // Its four neighbours above carried `$options: 'i'` until #6682, which + // was them folding Unicode for the case-SENSITIVE `$contains` family. + // That flag is now spelled nowhere in this function, which is the shape + // to keep: `$options` is a RETIRED operator (#5702), and the only + // sanctioned case-insensitive answer on this driver is the pattern below. case '$icontains': result.$regex = asciiCaseInsensitiveRegexSource(String(value)); break; diff --git a/packages/drivers/driver-mongodb/src/mongodb-icontains.test.ts b/packages/drivers/driver-mongodb/src/mongodb-icontains.test.ts index bd68346a32..b95e8ec9f6 100644 --- a/packages/drivers/driver-mongodb/src/mongodb-icontains.test.ts +++ b/packages/drivers/driver-mongodb/src/mongodb-icontains.test.ts @@ -15,10 +15,17 @@ * right" into "the pattern selects the right rows". * * This file deliberately drives the shared text fixture's ROWS rather than - * naming that case-set's marker export: doing the latter would flip this - * driver's conformance cell to covered while requirement 2 (the `$contains` - * family's hardcoded `$options: 'i'`) is still open — #6682 — and an entry for a - * covered cell fails the gate's RECONCILED invariant. + * naming that case-set's marker export: doing the latter would have flipped + * this driver's conformance cell to covered while requirement 2 (the + * `$contains` family's hardcoded `$options: 'i'`) was still open — #6682 — and + * an entry for a covered cell fails the gate's RECONCILED invariant. + * + * [#6682] That requirement is answered now, and the marker is imported by + * `mongodb-filter-text-conformance.test.ts`, which is what CONSUMED counts and + * where the ledger row used to be. This file stays as it is rather than being + * folded into that one: it asserts the emitted PATTERN (`[Aa][Cc]…`, which the + * case-set cannot express — it speaks in row ids), and the spelled-out cases + * here are the ASCII-boundary reasoning in the form the #6520 card left it. */ import { describe, it, expect } from 'vitest'; @@ -47,15 +54,25 @@ describe('[#6520] $icontains translates to an ASCII-only case-insensitive $regex }); /** - * The contrast that makes the row above meaningful: the four case-EXACT - * operators still carry the hardcoded `$options: 'i'` this driver has always - * had. That is a DEFECT (#6682 — those four must be case-sensitive), pinned - * here as the current state so that "fixed the family" and "broke - * `$icontains`" cannot be confused for one another. + * The contrast that makes the row above meaningful, FLIPPED by #6682. + * + * This assertion used to pin the DEFECT — the four case-EXACT operators + * carrying the hardcoded `$options: 'i'` this driver had always had — so that + * "fixed the family" and "broke `$icontains`" could not be confused for one + * another. The family is fixed now, so the pin asserts the ruled answer + * rather than being deleted: `$contains` emits a bare pattern, `$icontains` + * emits the ASCII class expansion, and the two remain visibly DIFFERENT + * spellings — which is what this row has always guarded. Neither is + * `$options: 'i'`, the retired spelling this driver refuses on input (#5702). */ - it('does not disturb the $contains family — still `$options: i` (the open #6682 defect)', () => { + it('the $contains family is case-SENSITIVE beside it — a bare pattern, no $options (#6682)', () => { expect(translateFilter({ name: { $contains: 'acme' } })).toEqual({ - name: { $regex: 'acme', $options: 'i' }, + name: { $regex: 'acme' }, + }); + // …and the two arms have NOT collapsed into one answer: `$icontains` still + // folds ASCII case, `$contains` no longer folds anything. + expect(translateFilter({ name: { $icontains: 'acme' } })).toEqual({ + name: { $regex: '[Aa][Cc][Mm][Ee]' }, }); }); diff --git a/packages/spec/src/data/filter-text-conformance.ts b/packages/spec/src/data/filter-text-conformance.ts index 6e13478812..4d8a855f0d 100644 --- a/packages/spec/src/data/filter-text-conformance.ts +++ b/packages/spec/src/data/filter-text-conformance.ts @@ -51,13 +51,18 @@ * which is what `scripts/check-driver-conformance.mjs` counts as coverage. * - `driver-memory` and `driver-mongodb` ANSWER `$icontains` since #6520 — on * all three of driver-memory's faces — with the same ASCII-only fold, so the - * first four rows of this table are satisfied everywhere. They still fold the - * `$contains` family over the whole Unicode range (#6682), which is why each - * still carries a measured DEBT row in that same gate's ledger rather than - * importing this table: coverage is judged by IMPORT, and a cell that answers - * one requirement and not the other must not claim the whole set. The ledger - * is RECONCILED against the imports on every run, so read the open set THERE - * rather than trusting a count written in prose here. + * first four rows of this table are satisfied everywhere. + * - `driver-mongodb` answers the `$contains` family case-exactly since #6682, + * and its suite (`mongodb-filter-text-conformance.test.ts`) imports this + * whole table — every row, rejections included — so its DEBT row is gone. + * - `driver-memory` still folds the `$contains` family over the whole Unicode + * range on its query and analytics faces while its reference matcher does + * not (#6682), which is why it still carries a measured DEBT row in that + * gate's ledger rather than importing this table: coverage is judged by + * IMPORT, and a cell that answers one requirement and not the other must not + * claim the whole set. The ledger is RECONCILED against the imports on every + * run, so read the open set THERE rather than trusting a count written in + * prose here. * * Rule 2 above still governs the open cells: the rows join a driver's suite * in the PR that closes its gap, not before. @@ -84,7 +89,7 @@ * @see https://github.com/objectstack-ai/objectstack/issues/5701 (this table) * @see https://github.com/objectstack-ai/objectstack/issues/5702 (the SQL family — landed) * @see https://github.com/objectstack-ai/objectstack/issues/6520 ($icontains on the JS faces — landed) - * @see https://github.com/objectstack-ai/objectstack/issues/6682 (the $contains family on memory + mongodb — open) + * @see https://github.com/objectstack-ai/objectstack/issues/6682 (the $contains family — mongodb landed, memory open) */ import type { FilterCondition } from './filter.zod'; @@ -249,16 +254,17 @@ export const FILTER_TEXT_CASES: readonly FilterTextCase[] = [ // are #5702\'s work" was the score when these rows landed and has since split // (re-measured 2026-08, #6993, by executing each face): #6518 made the SQL // family case-exact (GLOB on the SQLite dialects), so those three drivers - // answer these rows today, while mongo\'s `$options: 'i'` is STILL hardcoded - // (`translateFilter` lowers `$contains` to `$regex` + `$options: 'i'`) and - // driver-memory\'s query path still folds Unicode — that remainder is #6682\'s - // work now, not #5702\'s. (`formula` and driver-memory\'s reference matcher - // measured case-exact both then and now.) + // answer these rows today, and #6682 took mongo\'s hardcoded `$options: 'i'` + // off all four arms, so `translateFilter` now lowers `$contains` to a bare + // `$regex` and that driver answers them too. driver-memory\'s query and + // analytics faces still fold Unicode — that remainder is #6682\'s open half, + // not #5702\'s. (`formula` and driver-memory\'s reference matcher measured + // case-exact both then and now.) { name: '$contains is case-SENSITIVE — a lower-case comparand misses the upper-case row', filter: { name: { $contains: 'acme' } }, expected: ['2'], - note: 'Row 1 (ACME Corp) must NOT match. SQLite\'s LIKE folds ASCII — the defect #6518 replaced with GLOB on the SQLite dialects; a backend returning both here has regressed to it (driver-memory / driver-mongodb still fold — #6682).', + note: 'Row 1 (ACME Corp) must NOT match. SQLite\'s LIKE folds ASCII — the defect #6518 replaced with GLOB on the SQLite dialects; a backend returning both here has regressed to it (driver-memory still folds — #6682).', }, { name: '$contains is case-SENSITIVE — an upper-case comparand misses the lower-case row', diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 81088eb512..735ab016cc 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -415,14 +415,18 @@ export const RangeOperatorSchema = lazySchema(() => z.object({ * * The `$contains`-family alignment the ruling above requires is likewise part * done rather than pending: #6518 made that family case-EXACT across the SQL - * dialects, while `driver-memory`'s query path and `driver-mongodb` still fold - * the whole Unicode range — the two rows #6682 tracks. + * dialects and #6682 did the same on `driver-mongodb` (the hardcoded + * `$options: 'i'` is off all four arms, and that driver's every face — query, + * count, write and the aggregation `$match` — routes through the one + * `translateFilter`, so there is no second answer). `driver-memory`'s query and + * analytics faces still fold the whole Unicode range while its reference + * matcher does not — the one row #6682 still tracks, and the one package that + * answers this operator two ways. * * `FILTER_TEXT_CASES` (`filter-text-conformance.ts`) is the standard that * measures all of the above, and the driver-conformance ledger still carries a - * DEBT row for `driver-memory` and `driver-mongodb` — on requirement 2 alone - * now, since #6520 closed requirement 1 on both — so what is open stays counted - * rather than assumed. + * DEBT row for `driver-memory` — on requirement 2 alone now, since #6520 closed + * requirement 1 — so what is open stays counted rather than assumed. * * @see FILTER_TEXT_CASES — the conformance standard for every operator here. * @see RETIRED_FILTER_OPERATORS — why `$regex` is not in this list. diff --git a/scripts/check-driver-conformance.mjs b/scripts/check-driver-conformance.mjs index 5ef431853f..9b8d624936 100644 --- a/scripts/check-driver-conformance.mjs +++ b/scripts/check-driver-conformance.mjs @@ -273,6 +273,19 @@ const CASE_SETS = [ // TEMPORAL_CASES, opt-in only — `mongodb-temporal-conformance.test.ts` // TEMPORAL_TIME_CASES has no server-free half. // +// Two markers joined that column after the day it was measured, and both joined +// it SERVER-FREE, which is the shape this note asks for rather than a way around +// it — the driver's answer to each is a pure translation, so a suite that +// evaluates the emitted document tests the driver and not MongoDB: +// +// AGGREGATION_CASES [#6850/#6814] `mongodb-aggregation-translation. +// test.ts` runs the case-set by default. The +// real-mongod half is absent, recorded on #6814. +// FILTER_TEXT_CASES [#6682] `mongodb-filter-text-conformance.test.ts` +// runs the case-set by default, rejection rows +// included. The real-mongod half is absent, +// recorded on #6682. +// // Why: on a cold binary cache two vitest workers downloaded the same ~123 MB // archive and the loser's `rename` blew up an all-green run as an unhandled // rejection, ejecting unrelated PRs from the merge queue. The maintainer retired @@ -280,16 +293,29 @@ const CASE_SETS = [ // investment is frozen (#5499). Un-freezing it is what should re-run these cells // in CI; until then, this note is the honest state of the mongo column. -// ## FILTER_TEXT_CASES: two DEBT rows left of the five #5701 opened +// ## FILTER_TEXT_CASES: ONE DEBT row left of the five #5701 opened // // The ledger was EMPTY (see the note above) until `FILTER_TEXT_CASES` arrived. // Those five rows were not a regression in coverage: the case-set is the // CONTRACT half of the #4706 ruling, landed deliberately ahead of every // implementation, and one row per driver is what made "ahead of" a counted fact // instead of an assumption. #5702 cleared two of the three requirements; #6518 -// cleared the third on the SQL family and DELETED its three rows. What remains -// is the #5499 frozen family — driver-memory and driver-mongodb — where the -// freeze, not the difficulty, is why the cells are open. +// cleared the third on the SQL family and DELETED its three rows. +// +// [#6682] `driver-mongodb`'s row is GONE too — the maintainer unfroze that +// package on 2026-08-11 (#5499), requirement 2 landed there (the hardcoded +// `$options: 'i'` is off all four `$contains`-family arms), and +// `mongodb-filter-text-conformance.test.ts` imports the marker and drives all +// SEVENTEEN cases including every rejection row. Measured rather than argued: +// on `origin/main` @ `744b8f5` that suite failed exactly the five +// case-sensitivity rows and passed the other twelve, so the fix was the whole +// remaining gap and nothing else was quietly widened to reach green. It is the +// SERVER-FREE half #5517 requires — this package's real-mongod suites are +// opt-in, so a suite needing a server would not run in CI — and it evaluates +// the emitted documents rather than pinning their spelling. +// +// What remains is `driver-memory`, still in the #5499 frozen family, where the +// freeze rather than the difficulty is why the cell is open. // // What the case-set demands, and where each requirement stands: // @@ -315,11 +341,15 @@ const CASE_SETS = [ // `LIKE` is already case-exact), and `LIKE` over `CAST(… AS BINARY)` on // MySQL (whose answer otherwise follows the column's collation). turso's // remote transport carries the twin in `pushLike`, and the two are held to -// the same rows by `turso-local-remote-text-parity.test.ts`. **STILL OPEN -// on driver-memory and driver-mongodb**, which fold the full Unicode range -// on their live query paths — and on driver-memory the REFERENCE matcher -// answers the same operator case-sensitively, so that package disagrees -// with itself. Both are the #5499 frozen family; tracked as #6682. +// the same rows by `turso-local-remote-text-parity.test.ts`. **DONE on +// driver-mongodb too** (#6682): `translateFieldOperators` no longer sets +// `$options: 'i'` on any of the four arms, and because every face of that +// driver — `find`/`count`/`update`/`delete` and the aggregation `$match` +// — routes through the one `translateFilter`, there is no second answer to +// align. **STILL OPEN on driver-memory**, which folds the full Unicode +// range on its live query path while its REFERENCE matcher answers the +// same operator case-sensitively, so that package disagrees with itself. +// It is the remaining #5499 frozen half; tracked as #6682. // // Two faces #6518 measured and did NOT have to change, recorded because // "not mentioned" reads as "not checked": `formula`'s `matchesFilter` and @@ -407,35 +437,6 @@ const LEDGER = [ + 'requirement 2 is open.', issue: 'https://github.com/objectstack-ai/objectstack/issues/6682', }, - { - driver: 'driver-mongodb', - marker: 'FILTER_TEXT_CASES', - kind: 'DEBT', - why: - 'Re-measured after #6518, which cleared requirement 2 on the SQL family and NOT here — #5499 freezes ' - + 'this package, so the cell stays open by decision rather than by difficulty. Still the FURTHEST from ' - + 'the ruling, but the ENVELOPE half is closed (#5702): that arm used to throw a bare ' - + "`new Error('[mongodb] unsupported filter operator …')` — no `code`, no `status` — three lines from " - + 'this file\'s own `unsupportedFilterError` helper, which sets `INVALID_FILTER` / 400 and which three ' - + 'other refusals here already used. It now routes through the helper, and a RETIRED spelling ' - + 'additionally gets the spec prescription naming `$icontains`, so requirement 3 is DONE (mongo was ' - + 'already the only backend REFUSING `$regex`; what was missing was the shape of the refusal). ' - + 'Requirement 2 is inverted here and requirement 1\'s ASCII boundary violated in the same expression: ' - + '`translateFieldOperators` lowers `$contains`/`$startsWith`/`$endsWith`/`$notContains` to `$regex` ' - + 'with a HARDCODED `$options: "i"` — tracked as #6682, the successor #6518 left behind for this pair ' - + 'of frozen packages. `escapeRegex` does escape metacharacters, so the literal-comparand cases hold. ' - + 'Requirement 1 is DONE here since #6520: `translateFieldOperators` has a `$icontains` arm, and it is ' - + 'the ONE arm in that family that does not set `$options: "i"` — the fold lives in the pattern ' - + '(`asciiCaseInsensitiveRegexSource`, one `[Aa]` class per ASCII letter), because mongo\'s `i` flag ' - + 'folds the whole Unicode range and would fail the CAFÉ rows. The non-empty-string comparand rule ' - + 'sits on the validating WALK beside `$null`\'s, not in the emitter, so it cannot be skipped by a ' - + 'boolean identity settling the enclosing node. So this row now carries ONE open requirement, not ' - + 'two. It still cannot go: ' - + 'coverage is judged by importing the whole case-set, so a half-answered cell must not import it. Note ' - + 'this package is in the #5499 frozen family: its real-mongod suites are opt-in, so whatever clears ' - + 'this cell needs a server-free half like `mongodb-filter-logic-translation.test.ts` has.', - issue: 'https://github.com/objectstack-ai/objectstack/issues/6682', - }, { driver: 'driver-memory', marker: 'AGGREGATION_CASES',