From af2436ed8c7a649fafe72d44b3b7a4e65f296aa4 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 11 Aug 2026 06:23:02 +0000 Subject: [PATCH 1/4] fix(spec): stop folding like/ilike onto $contains at the wire (#7536) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The wire lowering carried 'like': '$contains' in AST_OPERATOR_MAP, so every like predicate arriving over HTTP was rewritten into a substring search before any driver saw it. $contains LIKE-escapes its comparand and wraps it in %…%, which broke like in both directions: a caller's wildcards bound as literals (['name','like','%Industries'] returned 0 rows) and a wildcard-free pattern became a substring match byte-identical to the $contains control. canonicalAstOperator already documented the contract being violated, thirty lines below the map entry, in a hand-written exemption for like/ilike. That exemption only shaped its own output; the lowering the wire takes had none. - spec: new $like/$ilike operators, the pattern language defined once (hasDanglingLikeEscape, likePatternToRegexSource, matchesLikePattern, likePatternToGlobPattern), like/ilike lowered to them, the exemption retired. - driver-sql: the emitter arm the wire could not reach since #5158 — LIKE on Postgres/MySQL, GLOB on SQLite (case-exactness, #6518), pattern translated rather than escaped. - driver-turso: the same on the remote transport, sharing the spec translation. - driver-memory: both faces, so the in-memory double does not 400 for a filter production answers; the stale infix like==contains arm is gone. - formula: arms, so a write-side check agrees with the read-side SQL. - client: contains/startsWith/endsWith stop gluing wildcards into a like tuple. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01PhHptz16p1kRmbmuzEZkgd --- .changeset/like-wire-lowering.md | 114 ++++++++ content/docs/api/client-sdk.mdx | 14 +- content/docs/deployment/troubleshooting.mdx | 22 +- packages/client/src/client.test.ts | 41 ++- packages/client/src/query-builder.ts | 65 ++++- .../driver-memory/src/filter-refusal.ts | 95 ++++++ .../driver-memory/src/memory-driver.ts | 52 +++- .../src/memory-like-pattern.test.ts | 183 ++++++++++++ .../driver-memory/src/memory-matcher.ts | 26 +- .../src/sql-driver-like-pattern.test.ts | 260 +++++++++++++++++ packages/drivers/driver-sql/src/sql-driver.ts | 211 +++++++++++++- .../driver-turso/src/remote-transport.ts | 107 +++++++ .../turso-local-remote-like-parity.test.ts | 150 ++++++++++ .../formula/src/matches-filter-like.test.ts | 92 ++++++ packages/formula/src/matches-filter.ts | 43 ++- packages/spec/authorable-surface/data.json | 2 + .../data/filter-like-wire-lowering.test.ts | 274 ++++++++++++++++++ .../data/filter-operator-vocabulary.test.ts | 32 +- packages/spec/src/data/filter.test.ts | 17 +- packages/spec/src/data/filter.zod.ts | 244 +++++++++++++++- 20 files changed, 2006 insertions(+), 38 deletions(-) create mode 100644 .changeset/like-wire-lowering.md create mode 100644 packages/drivers/driver-memory/src/memory-like-pattern.test.ts create mode 100644 packages/drivers/driver-sql/src/sql-driver-like-pattern.test.ts create mode 100644 packages/drivers/driver-turso/src/turso-local-remote-like-parity.test.ts create mode 100644 packages/formula/src/matches-filter-like.test.ts create mode 100644 packages/spec/src/data/filter-like-wire-lowering.test.ts diff --git a/.changeset/like-wire-lowering.md b/.changeset/like-wire-lowering.md new file mode 100644 index 0000000000..d4ddd91693 --- /dev/null +++ b/.changeset/like-wire-lowering.md @@ -0,0 +1,114 @@ +--- +"@objectstack/spec": minor +"@objectstack/driver-sql": minor +"@objectstack/driver-turso": minor +"@objectstack/driver-memory": minor +"@objectstack/formula": minor +"@objectstack/client": patch +--- + +fix(spec,drivers,formula,client): `like`/`ilike` stop being folded onto `$contains` at the wire (#7536) + +A `like` predicate that arrived over HTTP was rewritten into a substring search +before any driver saw it, because `AST_OPERATOR_MAP` (`data/filter.zod.ts`) +carried `'like': '$contains'`. `$contains` LIKE-escapes its comparand and wraps +it in `%…%`, which breaks a `like` in **both** directions at once. Measured in +QA run #7463 against showcase on SQLite: + +| filter | before | now | +|---|---|---| +| `["name","like","%Industries"]` | `200`, **0 rows** — the `%` bound as a literal percent sign | the rows ENDING WITH `Industries` | +| `["name","like","Industries"]` | a substring match, **byte-identical to the `$contains` control** | an EXACT match | +| `["name","ilike","…"]` | `400` — `ilike` had no lowering at all, so `isFilterAST()` refused the whole filter | the case-insensitive twin | + +The second row is the tell: `like` and `$contains` producing the same bytes +means `like` was not reaching the driver as a pattern at all. + +The file already documented the contract being violated. `canonicalAstOperator`, +thirty lines below the map entry, carried a hand-written exemption for +`like`/`ilike` whose comment read: *"they are NOT substring matches at the +driver: driver-sql passes them to SQL verbatim, so the caller binds the +wildcards. Folding them onto `contains` would silently wrap the value in `%…%` +and change what the query means."* That exemption only ever shaped its own +output; the lowering the wire path takes had none. A consequence worth naming: +driver-sql's `like`/`ilike` handling has been unreachable from the wire since +#5158. + +## What changed + +**New operators `$like` / `$ilike`** on `StringOperatorSchema` and +`FieldOperatorsSchema`. The comparand IS the pattern: `%` matches any sequence, +`_` matches exactly one character, a backslash escapes either, and the pattern +must cover the WHOLE value — so a pattern with no wildcards is an exact +comparison, not a substring search. `$like` is case-SENSITIVE (the #4706 Q2 = A +contract its `$contains` sibling answers); `$ilike` folds ASCII case and nothing +else (Q1 = A), so `café` does not match `CAFÉ`. + +`AST_OPERATOR_MAP` now lowers `like` → `$like` and `ilike` → `$ilike`. `ilike` +enters the AST vocabulary for the first time — it previously had no entry, so +`isFilterAST()` refused it. `canonicalAstOperator`'s hand-written exemption is +retired: the generic round-trip answers `like`/`ilike` by construction now, so +the special case is gone along with the reason it existed. + +The pattern language is defined **once**, in the spec, and shared by every face +that needs it — `hasDanglingLikeEscape`, `likePatternToRegexSource`, +`matchesLikePattern` and `likePatternToGlobPattern`. Six faces implementing one +pattern language separately is the `#3948` shape reached through translation +instead of vocabulary. + +**Which backends answer, and which refuse.** `$like`/`$ilike` are deliberately +NOT in `FILTER_OPERATORS`, the runtime allowlist several packages derive +acceptance from — adding a name there before every face has an arm turns a loud +refusal into a silently DROPPED predicate, which is the widening measured in +#5701 and ruled on in #3948. + +| face | `$like` / `$ilike` | +|---|---| +| `driver-sql` (and `driver-sqlite-wasm`, which inherits its compiler) | **answers** — `LIKE` on Postgres/MySQL, `GLOB` on SQLite | +| `driver-turso`, both transports | **answers** — the remote transport compiles independently, holds to the local one by a parity suite | +| `driver-memory`, both faces | **answers** — the in-memory double must not 400 for a filter that works in production | +| `@objectstack/formula` (`matchesFilterCondition`) | **answers** — so a write-side RLS `check` agrees with the read-side SQL | +| `driver-mongodb`, objectql `having`, `service-analytics` | **refuse**, loudly, in the ADR-0112 `INVALID_FILTER` envelope | + +The refusals are the point rather than a gap: #7536 exists because a `like` was +silently given `$contains`' meaning, and a face that quietly answers a different +question is worse than one that refuses. Clearing the remainder means arms on +those faces in one PR — the #6520 direction. + +**Why SQLite gets `GLOB`.** `$like` is case-exact and SQLite's `LIKE` folds +ASCII unconditionally, which cannot be switched off per statement +(`PRAGMA case_sensitive_like` is connection-global). That is #6518's finding, +and the operator it landed on. Because GLOB speaks a different pattern language +(`*`/`?`, and `%`/`_` are ordinary characters), the pattern is TRANSLATED rather +than escaped — including GLOB's own metacharacters, which are ordinary to LIKE: +an unescaped `*` in a GLOB pattern is the same filter bypass an unescaped `%` is +under LIKE (#5567). + +**Refused rather than given a meaning:** a pattern ending in a lone unpaired +backslash. No reading survives every backend — Postgres rejects such a pattern +outright, GLOB has no escape character at all — so it is refused at the door on +every face, by one shared test. + +## ⚠️ Behaviour changes + +1. **`like` now means `LIKE`.** If you were relying on `like` behaving as a + substring search — the defect — write `contains` instead. A wildcard-free + `like` is now an exact match. +2. **`like`/`ilike` on `driver-mongodb`, objectql `having` and analytics now + return `400 INVALID_FILTER`** where a (wrong) substring answer came back + before. Write `$contains`/`$icontains` on those backends. +3. **`@objectstack/client`'s `.contains()`, `.startsWith()` and `.endsWith()` + emit different operators.** They used to build a `like` tuple by gluing + wildcards onto the caller's value (`[field, 'like', '%' + value + '%']`), + which was wrong twice over: the wire folded `like` onto `$contains`, which + escaped the glued `%` back into a literal, so `.contains('name','Corp')` + searched for the text `%Corp%` and matched only rows containing percent + signs. And once `like` reaches the driver as a real pattern, the glue becomes + the *other* bug — a `%` or `_` inside the caller's own value would silently + become a wildcard. They now emit `contains` / `starts_with` / `ends_with`, + whose comparand is text. `.like()` is unchanged and finally works; `.ilike()` + is new. + + Note the case semantics this corrects on paper too: `.contains()`'s docblock + claimed "case-insensitive", but the `$contains` family is case-SENSITIVE by + contract (#4706 Q2 = A). Use `.ilike()` for a case-insensitive pattern. diff --git a/content/docs/api/client-sdk.mdx b/content/docs/api/client-sdk.mdx index 4bdaaf52f7..6293cb91af 100644 --- a/content/docs/api/client-sdk.mdx +++ b/content/docs/api/client-sdk.mdx @@ -473,14 +473,22 @@ const filter = createFilter() .greaterThan('revenue', 10000) // field > value .lessThanOrEqual('age', 100) // field <= value .in('category', ['A', 'B', 'C']) // field IN (...) - .like('name', '%Corp%') // field LIKE pattern - .contains('name', 'Corp') // LIKE %Corp% - .startsWith('name', 'Acme') // LIKE Acme% + .like('name', '%Corp%') // pattern match — the wildcards are YOURS + .ilike('name', '%corp%') // same, ignoring ASCII case + .contains('name', 'Corp') // literal substring — no wildcards involved + .startsWith('name', 'Acme') // literal prefix .isNull('deleted_at') // field IS NULL .between('created_at', '2024-01', '2024-12') .build(); ``` +`like` / `ilike` take a **pattern**: `%` matches any sequence, `_` matches +exactly one character, and a backslash escapes either. The pattern is matched +against the whole value, so `like('name', 'Corp')` is an exact comparison — for +a substring search use `contains`, whose argument is plain text and needs no +escaping. `like` / `ilike` are executed by the SQL backends; the others refuse +them with `invalid_filter`. + --- ## Query Options diff --git a/content/docs/deployment/troubleshooting.mdx b/content/docs/deployment/troubleshooting.mdx index 0ee52a786f..e47c57a0eb 100644 --- a/content/docs/deployment/troubleshooting.mdx +++ b/content/docs/deployment/troubleshooting.mdx @@ -125,7 +125,8 @@ The custom error map provides "Did you mean?" suggestions for common typos. ``` $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $between, -$contains, $notContains, $startsWith, $endsWith, +$contains, $notContains, $startsWith, $endsWith, $icontains, +$like, $ilike, $null, $exists, $and, $or, $not ``` @@ -135,10 +136,27 @@ Common mistakes: |:---|:---| | `$equal` | `$eq` | | `$notEqual` | `$ne` | -| `$like` | `$contains` | | `$greaterThan` | `$gt` | | `$isNull` | `$null` | | `$notIn` | `$nin` | +| `$regex` | `$icontains` | + +`$like` is **not** a misspelling of `$contains` — the two mean different things, +and picking the wrong one gives a silently wrong answer rather than an error: + +- `$contains` takes **text**, matched literally as a substring. A `%` or `_` in + the comparand is an ordinary character. +- `$like` takes a **pattern**, matched against the whole value, with the + wildcards you write: `%` is any sequence, `_` is exactly one character, and a + backslash escapes either. A pattern with no wildcards is an exact comparison, + not a substring search. + +`$ilike` is `$like`'s case-insensitive twin, and `$icontains` is `$contains`'s. +Both fold ASCII case only, so `café` does not match `CAFÉ`. + +`$like` / `$ilike` are executed by the SQL family (`driver-sql`, +`driver-sqlite-wasm`, `driver-turso`). The other backends refuse them with this +same `invalid_filter` error — write `$contains` there. --- diff --git a/packages/client/src/client.test.ts b/packages/client/src/client.test.ts index bf8cd244c1..298a2a9ad6 100644 --- a/packages/client/src/client.test.ts +++ b/packages/client/src/client.test.ts @@ -1015,25 +1015,58 @@ describe('FilterBuilder enhancements', () => { expect(f[2]).toEqual(['age', '<=', 65]); }); - it('should add contains filter', () => { + // [#7536] These three used to build a `like` tuple by gluing wildcards onto + // the caller's value (`['name', 'like', '%alice%']`), which was wrong in two + // directions at once. While the wire folded `like` onto `$contains` — which + // LIKE-ESCAPES its comparand — the glued `%` came back as a literal percent + // sign, so `.contains('name','alice')` searched for the text `%alice%` and + // matched only rows containing percent signs. And once `like` reaches the + // driver as a real pattern, the glue becomes the OTHER bug: a `%` or `_` + // inside the caller's own value would silently become a wildcard. Naming the + // operator that means what the method says removes both. + it('should add contains filter as the literal-text operator', () => { const f = createFilter<{ name: string }>() .contains('name', 'alice') .build(); - expect(f).toEqual(['name', 'like', '%alice%']); + expect(f).toEqual(['name', 'contains', 'alice']); + }); + + it('should not let the caller\'s own wildcards leak into a contains filter', () => { + // The regression this shape prevents: `50%` is TEXT here, not a pattern. + const f = createFilter<{ name: string }>() + .contains('name', '50%') + .build(); + expect(f).toEqual(['name', 'contains', '50%']); }); it('should add startsWith filter', () => { const f = createFilter<{ name: string }>() .startsWith('name', 'A') .build(); - expect(f).toEqual(['name', 'like', 'A%']); + expect(f).toEqual(['name', 'starts_with', 'A']); }); it('should add endsWith filter', () => { const f = createFilter<{ email: string }>() .endsWith('email', '.com') .build(); - expect(f).toEqual(['email', 'like', '%.com']); + expect(f).toEqual(['email', 'ends_with', '.com']); + }); + + it('should pass a like() pattern through UNCHANGED — the wildcards are the caller\'s', () => { + // The one method that always meant "pattern", and the one the wire + // lowering broke: `%Industries` matched nothing before #7536. + const f = createFilter<{ name: string }>() + .like('name', '%Industries') + .build(); + expect(f).toEqual(['name', 'like', '%Industries']); + }); + + it('should add ilike filter for a case-insensitive pattern', () => { + const f = createFilter<{ name: string }>() + .ilike('name', '%industries') + .build(); + expect(f).toEqual(['name', 'ilike', '%industries']); }); it('should add exists filter', () => { diff --git a/packages/client/src/query-builder.ts b/packages/client/src/query-builder.ts index a93c6159de..5d10fb18af 100644 --- a/packages/client/src/query-builder.ts +++ b/packages/client/src/query-builder.ts @@ -83,7 +83,19 @@ export class FilterBuilder { } /** - * LIKE filter: field LIKE pattern + * LIKE filter: `field LIKE pattern`, with the wildcards YOU write. + * + * `%` matches any sequence, `_` matches exactly one character, and a + * backslash escapes either. The pattern must cover the WHOLE value, so a + * pattern with no wildcards is an exact comparison — use {@link contains} + * for a substring search. + * + * [#7536] This is the method that did not work. The wire lowering folded + * every `like` spelling onto `$contains`, so the pattern was LIKE-escaped and + * wrapped in `%…%`: `.like('name', '%Industries')` matched NOTHING (the `%` + * bound as a literal percent sign) and `.like('name', 'Industries')` returned + * a substring match indistinguishable from `.contains(...)`. It now reaches + * the driver as a real pattern. */ like(field: K, pattern: string): this { this.conditions.push([field as string, 'like', pattern]); @@ -115,26 +127,65 @@ export class FilterBuilder { } /** - * CONTAINS filter: field contains value (case-insensitive LIKE %value%) + * CONTAINS filter: the field's text contains `value`, compared + * case-SENSITIVELY and LITERALLY. + * + * [#7536] These three methods used to build a `like` tuple by gluing + * wildcards onto the caller's value — `[field, 'like', '%' + value + '%']` — + * and that was wrong in two directions at once, both of them silent: + * + * - The wire folded `like` onto `$contains`, which LIKE-escapes its comparand. + * So the glued `%` was escaped back into a literal percent sign and + * `.contains('name', 'Corp')` searched for the text `%Corp%` — matching only + * rows that literally contain percent signs. + * - Once `like` reaches the driver as a real pattern (this issue's fix), the + * glue becomes the OTHER bug: a caller's own `%` or `_` inside `value` + * would silently become a wildcard, so `.startsWith('name', 'a_b')` would + * also match `axb`. + * + * Both disappear by naming the operator that means what these methods say. + * `contains` / `starts_with` / `ends_with` are declared for exactly this, and + * their comparand is TEXT — matched literally, with no escaping burden on the + * caller. Note the case: the `$contains` family is case-SENSITIVE by contract + * (#4706 Q2 = A), which this method's docblock used to claim otherwise; for a + * caller-written pattern use {@link like}, and for case-insensitive matching + * use {@link ilike}. */ contains(field: K, value: string): this { - this.conditions.push([field as string, 'like', `%${value}%`]); + this.conditions.push([field as string, 'contains', value]); return this; } /** - * STARTS WITH filter: field starts with value (LIKE value%) + * ILIKE filter: `field ILIKE pattern` — {@link like}'s case-insensitive twin. + * + * Same pattern language and the same caller-bound wildcards; the fold is + * ASCII-only (`A-Z` against `a-z`), so `café` does NOT match `CAFÉ`. That + * boundary is the protocol's (#4706 Q1 = A), because SQLite folds ASCII only + * and three of the five backends are SQLite underneath. + */ + ilike(field: K, pattern: string): this { + this.conditions.push([field as string, 'ilike', pattern]); + return this; + } + + /** + * STARTS WITH filter: the field's text starts with `value`, compared + * case-SENSITIVELY and LITERALLY. See {@link contains} for why this no longer + * builds a `like` pattern. */ startsWith(field: K, value: string): this { - this.conditions.push([field as string, 'like', `${value}%`]); + this.conditions.push([field as string, 'starts_with', value]); return this; } /** - * ENDS WITH filter: field ends with value (LIKE %value) + * ENDS WITH filter: the field's text ends with `value`, compared + * case-SENSITIVELY and LITERALLY. See {@link contains} for why this no longer + * builds a `like` pattern. */ endsWith(field: K, value: string): this { - this.conditions.push([field as string, 'like', `%${value}`]); + this.conditions.push([field as string, 'ends_with', value]); return this; } diff --git a/packages/drivers/driver-memory/src/filter-refusal.ts b/packages/drivers/driver-memory/src/filter-refusal.ts index b80f211293..e40ba334c0 100644 --- a/packages/drivers/driver-memory/src/filter-refusal.ts +++ b/packages/drivers/driver-memory/src/filter-refusal.ts @@ -31,6 +31,9 @@ */ import { FILTER_OPERATORS, LOGICAL_OPERATORS, RETIRED_FILTER_OPERATORS } from '@objectstack/spec/data'; +// [#7536] The `$like` pattern language's shared gate, so this driver refuses +// the same malformed patterns as every other face. +import { hasDanglingLikeEscape } from '@objectstack/spec/data'; import { StandardErrorCode } from '@objectstack/spec/api'; /** @@ -182,6 +185,33 @@ export function emptyFieldConstraintError(field: string, path: string): Error { * entry in `FILTER_OPERATORS` is a claim that this driver evaluates it, and this * file will make that claim on the spec's behalf whether or not it is true. * + * ## [#7536] `$like` / `$ilike` arrive here EXPLICITLY, which is the other risk + * + * They are the mirror image of the paragraph above: declared by + * `StringOperatorSchema`, deliberately staged OUT of `FILTER_OPERATORS` (see its + * note) because `driver-mongodb`, objectql's `having` and `service-analytics` + * have no arm for them — and added to this set BY HAND, because this driver + * does. The precedent is `driver-turso`'s remote transport, whose + * `SUPPORTED_FILTER_OPERATORS` has carried `...FILTER_OPERATORS, '$icontains'` + * since #5702 for exactly this situation: a backend that implements one more + * operator than the shared allowlist says so LOCALLY, rather than pushing the + * allowlist ahead of the backends that cannot follow. + * + * Why this driver implements rather than refuses, when two of its siblings + * refuse: it is the in-memory DOUBLE. An application whose tests run here and + * whose production runs SQL would otherwise get a 400 in test for a filter that + * works in production — which is the same "one filter, two answers" divergence + * #6520 closed, wearing a refusal instead of a wrong row set. It is also the + * one driver holding the `VALID_AST_OPERATORS` expressibility invariant + * (`memory-filter-ast-vocabulary.test.ts`, #3948): every operator the protocol + * PARSES must survive to a matched row here. + * + * The ordering rule from the `$icontains` paragraph applies unchanged and was + * followed: both arms (`memory-driver.ts`'s query path and `memory-matcher.ts`) + * landed in the same commit as this widening. A name added here with no arm + * behind it is the #5701 measurement — gate stops refusing, matcher has no + * case, predicate silently DROPPED, every row matches. + * * Everything else is refused. That includes the mingo operators this driver used * to hand through by accident (`$elemMatch`, `$size`, `$type`, `$mod`, `$where`, * `$expr`, field-level `$not`) — none of them is in the Filter Protocol, none is @@ -189,6 +219,8 @@ export function emptyFieldConstraintError(field: string, path: string): Error { */ export const SUPPORTED_FIELD_OPERATORS: ReadonlySet = new Set([ ...FILTER_OPERATORS, + '$like', + '$ilike', ]); /** The vocabulary as it appears in a refusal message, in declaration order. */ @@ -643,6 +675,22 @@ function assertFieldConstraintShape( if (op === '$icontains' && (typeof spec[op] !== 'string' || spec[op] === '')) { throw icontainsComparandError(field, spec[op], `${path}.$icontains`); } + // [#7536] `$like` / `$ilike` carry a PATTERN. Two rules, both of them + // driver-sql's word for word so a suite that swaps this driver for SQL sees + // the same refusal for the same input. + // + // Note what is deliberately NOT refused: an EMPTY pattern. `$icontains: ''` + // is refused just above because every row contains the empty substring — + // but `LIKE ''` matches only the empty string, a narrow and well-formed + // predicate. Copying the neighbour's rule would refuse a legitimate query. + if (op === '$like' || op === '$ilike') { + if (typeof spec[op] !== 'string') { + throw likePatternComparandError(field, op, spec[op], `${path}.${op}`); + } + if (hasDanglingLikeEscape(spec[op] as string)) { + throw danglingLikeEscapeError(field, op, spec[op] as string, `${path}.${op}`); + } + } } // [#5702] The `$options`-without-`$regex` companion check that stood here is // GONE. It was needed while `$options` was an allowlisted MODIFIER — a key the @@ -667,6 +715,53 @@ function assertFieldConstraintShape( * constrains nothing. A dropped predicate WIDENS a result set, and on an RLS * read scope that is a permission bypass rather than a degraded filter. */ +/** + * [#7536] `$like` / `$ilike` received a comparand that is not a string. + * + * `driver-sql`'s `likePatternComparandError`, word for word, for the reason its + * `$icontains` neighbour gives. Coercion is worse for a pattern than for text: + * `String({})` is `[object Object]`, and once that reaches the SQL family's + * SQLite arm its `[` OPENS a GLOB character class — so the query would run a + * PATTERN nobody wrote rather than merely compare text nobody wrote. + */ +export function likePatternComparandError( + field: string, + op: string, + value: unknown, + path = 'filter', +): Error { + return unsupportedFilterError( + `Operator "${op}" on field "${field}" at ${path} requires a string comparand, received ` + + `${JSON.stringify(value) ?? String(value)}. "${op}" takes a PATTERN — "%" matches any ` + + `sequence, "_" matches one character, and a backslash escapes either — so a non-string ` + + `comparand cannot be coerced without inventing wildcards the caller never wrote. For a ` + + `literal substring search write "$contains", whose comparand IS text.`, + ); +} + +/** + * [#7536] A `$like` / `$ilike` pattern ending in a lone unpaired backslash. + * + * Refused rather than given a meaning because no meaning survives every + * backend: Postgres rejects such a pattern outright, SQLite's GLOB has no + * escape character at all, and a JS translation would have to invent a third + * answer. `hasDanglingLikeEscape` is the spec's shared test, so every face + * refuses the SAME patterns. + */ +export function danglingLikeEscapeError( + field: string, + op: string, + pattern: string, + path = 'filter', +): Error { + return unsupportedFilterError( + `Operator "${op}" on field "${field}" at ${path} has a pattern ending in a lone unpaired ` + + `backslash (${JSON.stringify(pattern)}). A backslash escapes the character after it, so a ` + + `trailing one escapes nothing and the backends disagree about what it means. Write ` + + `"\\\\\\\\" to match a literal backslash, or drop the trailing one.`, + ); +} + function icontainsComparandError(field: string, value: unknown, path: string): Error { const shown = typeof value === 'string' ? `""` : JSON.stringify(value) ?? String(value); return unsupportedFilterError( diff --git a/packages/drivers/driver-memory/src/memory-driver.ts b/packages/drivers/driver-memory/src/memory-driver.ts index 5177d9ceee..b0c31f8827 100644 --- a/packages/drivers/driver-memory/src/memory-driver.ts +++ b/packages/drivers/driver-memory/src/memory-driver.ts @@ -6,6 +6,10 @@ import type { DriverOptions } from '@objectstack/spec/data'; // strings, so the fold has to live in the pattern source. See its docblock for // why an `i` flag is the wrong tool. import { canonicalAstOperator, asciiCaseInsensitiveRegexSource } from '@objectstack/spec/data'; +// [#7536] `$like`/`$ilike`'s pattern language, from the spec's one definition — +// the same translation `formula` evaluates and the same pattern `driver-sql` +// hands to LIKE/GLOB, so this face cannot answer a pattern differently. +import { hasDanglingLikeEscape, likePatternToRegexSource } from '@objectstack/spec/data'; import type { DriverQuery, IDataDriver } from '@objectstack/spec/contracts'; import { Logger, createLogger, nextUtcCalendarDay } from '@objectstack/core'; import { Query, Aggregator } from 'mingo'; @@ -20,6 +24,9 @@ import { unknownFieldOperatorError, unknownLogicalOperatorError, unsupportedFilterError, + // [#7536] The `$like`/`$ilike` comparand refusals, beside their siblings. + likePatternComparandError, + danglingLikeEscapeError, } from './filter-refusal.js'; import { coerceTemporalValue, @@ -772,7 +779,8 @@ export class InMemoryDriver implements IDataDriver { // `VALID_AST_OPERATORS` accepts `>`, `gt`, `greater_than`, `greaterthan` and // `after` for the same thing. A private alias list here is what let this // driver and driver-sql accept different vocabularies. #3948. - switch (canonicalAstOperator(operator)) { + const canonical = canonicalAstOperator(operator); + switch (canonical) { case '=': case '==': return { [field]: store(value) }; case '!=': case '<>': @@ -796,8 +804,24 @@ export class InMemoryDriver implements IDataDriver { return { [field]: { $in: store(value) } }; case 'nin': case 'not_in': case 'notin': case 'not in': return { [field]: { $nin: store(value) } }; - case 'contains': case 'like': case 'ilike': + case 'contains': return { [field]: { $regex: new RegExp(this.escapeRegex(value), 'i') } }; + // [#7536] `like` / `ilike` are NOT `contains`, and sharing this arm with + // it was the memory-face twin of the wire defect #7536 closed: the + // comparand was regex-ESCAPED (so a caller's `%` matched a literal percent + // sign) and matched as a SUBSTRING (so a wildcard-free pattern matched + // anywhere in the value). Both readings answer a query nobody wrote. + // + // Now the same pattern translation the `$`-spelling takes one method over, + // so this driver answers one filter one way whichever door it came + // through (#3948). + case 'like': case 'ilike': { + if (typeof value !== 'string') throw likePatternComparandError(field, canonical, value); + if (hasDanglingLikeEscape(value)) throw danglingLikeEscapeError(field, canonical, value); + return { + [field]: { $regex: new RegExp(likePatternToRegexSource(value, canonical === 'ilike')) }, + }; + } case 'notcontains': case 'not_contains': return { [field]: { $not: { $regex: new RegExp(this.escapeRegex(value), 'i') } } }; case 'startswith': case 'starts_with': @@ -986,6 +1010,30 @@ export class InMemoryDriver implements IDataDriver { case '$icontains': regexConditions.push({ $regex: new RegExp(asciiCaseInsensitiveRegexSource(val)) }); break; + // [#7536] `$like` / `$ilike` — the caller's OWN pattern, anchored to the + // whole value. `likePatternToRegexSource` is the spec's one translation + // of that language, the same one `formula` evaluates and the same + // pattern `driver-sql` hands to `LIKE` / `GLOB`. + // + // Two things this arm must NOT copy from its neighbours above. It does + // not `escapeRegex` the comparand — a `%` here is the caller's wildcard, + // and escaping it back into a literal IS the wire defect #7536 closed, + // reproduced one layer down. And it does not pass the `i` flag: the + // `$ilike` fold is ASCII-only and lives in the pattern source, for the + // reason the `$icontains` arm above spells out at length. + case '$like': + case '$ilike': + // The comparand's shape was settled by `assertFieldConstraintShape` + // on the whole tree before this ran (the #5324/#5328 discipline every + // arm here follows), so `val` is a string with no dangling escape. + // The re-check is the totality floor a translator owes itself, the + // same one `driver-sql`'s emitter keeps beside its own gate. + if (typeof val !== 'string') throw likePatternComparandError(field, op, val, path); + if (hasDanglingLikeEscape(val)) throw danglingLikeEscapeError(field, op, val, path); + regexConditions.push({ + $regex: new RegExp(likePatternToRegexSource(val, op === '$ilike')), + }); + break; case '$between': { // [#5328] The arm used to be CONDITIONAL — a comparand that was not a // two-element array skipped it and wrote nothing, so the field diff --git a/packages/drivers/driver-memory/src/memory-like-pattern.test.ts b/packages/drivers/driver-memory/src/memory-like-pattern.test.ts new file mode 100644 index 0000000000..0d973a70d1 --- /dev/null +++ b/packages/drivers/driver-memory/src/memory-like-pattern.test.ts @@ -0,0 +1,183 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#7536] `$like` / `$ilike` on driver-memory — BOTH faces, one answer. + * + * ## Why this driver implements what two of its siblings refuse + * + * #7536 stopped the wire from folding `like` onto `$contains` and gave the SQL + * family a real pattern arm. The other backends then had a choice per the + * issue's own bar: implement the same semantics, or refuse loudly in the + * ADR-0112 envelope — never quietly answer something else. `driver-mongodb`, + * objectql's `having` and `service-analytics` refuse, because they have no arm + * and are in the #5499 frozen family. This driver implements, for two reasons + * that do not apply to them: + * + * 1. **It is the in-memory DOUBLE.** An application whose tests run here and + * whose production runs SQL would get a 400 in test for a filter that works + * in production. That is the same "one filter, two answers" divergence #6520 + * closed, wearing a refusal instead of a wrong row set. + * 2. **It holds the `VALID_AST_OPERATORS` expressibility invariant** (#3948, + * `memory-filter-ast-vocabulary.test.ts`): every operator the protocol + * PARSES must survive to a matched row here. `like` is in that set — and + * `ilike` joined it in #7536 — so a refusal would break the invariant rather + * than satisfy it. + * + * The semantics come from the spec's shared `matchesLikePattern` / + * `likePatternToRegexSource`, which is the same translation `formula` + * evaluates and the same pattern `driver-sql` hands to `LIKE` / `GLOB`. That + * sharing is the point: this package has TWO evaluation faces (the mingo query + * path and the reference matcher), and the divergence #5374 fixed for + * `$contains` between exactly those two faces is the failure mode here. + * + * `$like`/`$ilike` are deliberately NOT in the spec's `FILTER_OPERATORS`; this + * driver widens its own `SUPPORTED_FIELD_OPERATORS` by hand instead — the + * precedent `driver-turso`'s remote transport set for `$icontains` in #5702. + * These cases go red if that widening is ever removed without removing the + * arms, and `memory-filter-ast-vocabulary.test.ts` goes red if the arms are + * removed without the widening. + */ + +import { describe, it, expect, beforeEach } from 'vitest'; +import { InMemoryDriver } from './memory-driver.js'; +import { match } from './memory-matcher.js'; + +const TABLE = 'like_probe'; + +const ROWS = [ + { id: '1', name: 'Acme Industries' }, + { id: '2', name: 'Industries Ltd' }, + { id: '3', name: 'Industries' }, + { id: '4', name: 'ACME INDUSTRIES' }, + { id: '5', name: '100% match' }, + { id: '6', name: '100X match' }, + { id: '7', name: 'a_b' }, + { id: '8', name: 'axb' }, +] as const; + +interface WireBearingError extends Error { + code?: string; + status?: number; +} + +describe('[#7536] driver-memory — $like / $ilike on both faces', () => { + let driver: InMemoryDriver; + + beforeEach(async () => { + driver = new InMemoryDriver({ persistence: false }); + await driver.connect(); + for (const row of ROWS) await driver.create(TABLE, { ...row }); + }); + + /** Ids the LIVE QUERY PATH returns (mingo), ascending. */ + const queryIds = async (where: unknown): Promise => + (await driver.find(TABLE, { where: where as never })) + .map((r: Record) => String(r.id)) + .sort((a, b) => a.localeCompare(b)); + + /** Ids the REFERENCE MATCHER returns, ascending. */ + const matcherIds = (where: unknown): string[] => + ROWS.filter((r) => match(r, where)).map((r) => r.id).sort((a, b) => a.localeCompare(b)); + + const refusalOf = async (where: unknown): Promise => { + const err = await driver + .find(TABLE, { where: where as never }) + .then(() => null, (e: unknown) => e as WireBearingError); + if (!err) throw new Error(`expected a refusal for ${JSON.stringify(where)}, but it ran`); + return err; + }; + + it('seeded the fixture (the premise)', async () => { + expect(await queryIds({})).toEqual(['1', '2', '3', '4', '5', '6', '7', '8']); + }); + + /** + * Every case is asserted on BOTH faces, and the faces are compared to each + * other FIRST — a drift between them reads as a drift rather than as two + * unrelated wrong answers. + */ + const CASES: Array<[string, Record, string[]]> = [ + // The card's own repro table. + ['a wildcard pattern binds the caller\'s %', { name: { $like: '%Industries' } }, ['1', '3']], + ['a wildcard-free pattern is EXACT, not a substring', { name: { $like: 'Industries' } }, ['3']], + ['% at the end', { name: { $like: 'Industries%' } }, ['2', '3']], + // The pattern language. + ['_ matches exactly one character', { name: { $like: 'a_b' } }, ['7', '8']], + ['a backslash escapes _ back to a literal', { name: { $like: 'a\\_b' } }, ['7']], + ['a backslash escapes % back to a literal', { name: { $like: '100\\% match' } }, ['5']], + // Case, and the ASCII boundary. + ['$like is case-SENSITIVE', { name: { $like: 'Acme%' } }, ['1']], + ['$ilike folds ASCII case', { name: { $ilike: 'acme%' } }, ['1', '4']], + // The control: `$contains` must NOT have moved. + // + // The comparand is `Ltd` rather than the `Industries` every other row uses, + // and that is deliberate. Measured on this branch: `{ $contains: + // 'Industries' }` answers ['1','2','3','4'] on the QUERY path and + // ['1','2','3'] on the reference matcher, because the query path lowers + // `$contains` to a RegExp carrying the `i` flag while the matcher uses + // `String.includes`. That is this package's KNOWN open divergence — the + // #6682 DEBT row the driver-conformance ledger carries for exactly this + // pair of faces — and it predates #7536 by a long way. Steering the control + // around it keeps this file measuring what it is about; pinning either + // answer here would enshrine a defect or fail for a reason unrelated to + // `$like`. + ['the $contains control still matches substrings', { name: { $contains: 'Ltd' } }, ['2']], + ]; + + for (const [name, where, expected] of CASES) { + it(name, async () => { + const fromQuery = await queryIds(where); + const fromMatcher = matcherIds(where); + expect(fromMatcher, 'the reference matcher disagrees with the query path').toEqual(fromQuery); + expect(fromQuery).toEqual(expected); + }); + } + + it('answers the same rows as it would for a substring search ONLY when asked to', async () => { + // The defect, stated as an inequality: under the fold these two were the + // same query. This is the assertion that catches a regression that moves + // BOTH operators. + expect(await queryIds({ name: { $like: 'Industries' } })) + .not.toEqual(await queryIds({ name: { $contains: 'Industries' } })); + }); + + it('expresses the INFIX like/ilike spellings identically', async () => { + // The QueryAST comparison door. Until #7536 this arm was SHARED with + // `contains` — the memory-face twin of the wire defect: the comparand was + // regex-ESCAPED (so `%` matched a literal percent sign) and matched as a + // SUBSTRING. One driver must not answer two ways depending on which door a + // filter came through (#3948). + const viaInfix = await queryIds({ + type: 'comparison', field: 'name', operator: 'like', value: '%Industries', + }); + expect(viaInfix).toEqual(['1', '3']); + const viaInfixI = await queryIds({ + type: 'comparison', field: 'name', operator: 'ilike', value: '%industries', + }); + expect(viaInfixI).toEqual(['1', '3', '4']); + }); + + // ── Refusals: the shapes no backend can agree on ────────────────────────── + + it('refuses a non-string pattern in the ADR-0112 envelope', async () => { + const err = await refusalOf({ name: { $like: 42 } }); + expect(err.code).toBe('INVALID_FILTER'); + expect(err.status).toBe(400); + expect(err.message).toContain('$like'); + expect(err.message).toContain('$contains'); + }); + + it('refuses a pattern ending in a lone unpaired backslash', async () => { + const err = await refusalOf({ name: { $like: 'abc\\' } }); + expect(err.code).toBe('INVALID_FILTER'); + expect(err.status).toBe(400); + expect(err.message).toContain('backslash'); + }); + + it('accepts an EMPTY pattern — it is not the widening $icontains refuses', async () => { + // `$icontains: ''` is refused because every row contains the empty + // substring. `LIKE ''` matches only the empty string, which constrains + // plenty. Copying the sibling's rule here would refuse a legitimate query. + expect(await queryIds({ name: { $like: '' } })).toEqual([]); + }); +}); diff --git a/packages/drivers/driver-memory/src/memory-matcher.ts b/packages/drivers/driver-memory/src/memory-matcher.ts index 7ad3c9a159..a4952cfd0f 100644 --- a/packages/drivers/driver-memory/src/memory-matcher.ts +++ b/packages/drivers/driver-memory/src/memory-matcher.ts @@ -22,7 +22,10 @@ // [#6520] `$icontains`' ASCII-only fold, defined once in the spec and shared by // every JS evaluation face — see `foldAsciiCase`'s docblock for why it is not // re-implemented per package. -import { reduceFilterVerdict, asciiCaseInsensitiveContains } from '@objectstack/spec/data'; +// [#7536] `$like`/`$ilike`'s pattern language, likewise the spec's one +// definition — shared with this package's query path, `formula`, and the SQL +// family's emitters. +import { reduceFilterVerdict, asciiCaseInsensitiveContains, matchesLikePattern } from '@objectstack/spec/data'; import { assertFilterConditionShape } from './filter-refusal.js'; @@ -236,6 +239,27 @@ function checkCondition(value: any, condition: any): boolean { if (typeof value !== 'string' || typeof target !== 'string' || !asciiCaseInsensitiveContains(value, target)) return false; break; + // [#7536] `$like` / `$ilike` — the caller's own pattern, anchored to + // the WHOLE value. `matchesLikePattern` is the spec's shared + // translation: the same one this driver's live query path binds as a + // regex, and the same pattern `driver-sql` hands to LIKE/GLOB — so + // the two faces of this package cannot answer one pattern two ways + // (the divergence #5374 fixed for `$contains` in this same file). + // + // A malformed pattern cannot reach here — `filter-refusal.ts` + // refuses a dangling trailing escape on the shape walk, before + // evaluation starts. The guard stays because `match()` is also + // called directly by driver doubles, and a total function must stay + // total. + case '$like': + case '$ilike': + if (typeof value !== 'string' || typeof target !== 'string') return false; + try { + if (!matchesLikePattern(value, target, op === '$ilike')) return false; + } catch { + return false; + } + break; case '$null': // $null: true → value must be null/undefined; $null: false → value must not be null/undefined // diff --git a/packages/drivers/driver-sql/src/sql-driver-like-pattern.test.ts b/packages/drivers/driver-sql/src/sql-driver-like-pattern.test.ts new file mode 100644 index 0000000000..cc4998a212 --- /dev/null +++ b/packages/drivers/driver-sql/src/sql-driver-like-pattern.test.ts @@ -0,0 +1,260 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#7536] `$like` / `$ilike` on the SQL family — the EXECUTION half of the fix, + * against a real SQLite engine. + * + * ## What this file is proving + * + * The spec half (`filter-like-wire-lowering.test.ts`) pins that a wire `like` + * lowers to `$like` instead of being folded onto `$contains`. That is necessary + * and not sufficient: the reason the fold survived from #5158 to #7536 is that + * this driver had **no arm to reach**. The array-format emitter that once + * handled the infix `like` went away with the array dialect in #5158, leaving + * only the two infix spellings in `SCALAR_COMPARAND_OPERATORS` — a comparand + * gate for an operator nothing could emit. So a lowering fix with no emitter + * would have turned a silent wrong answer into a 400, and the card's repro + * table would still not pass. + * + * These cases therefore run rows, not SQL strings, and they run the card's own + * table: + * + * | filter | before #7536 | here | + * |---|---|---| + * | `["name","like","%Industries"]` | `200`, **0 rows** (the `%` bound as a literal) | the rows ENDING WITH `Industries` | + * | `["name","like","Industries"]` | a substring match, byte-identical to the `$contains` control | an EXACT match | + * | the `$contains` control | `%…%`-wrapped substring | unchanged — still `%…%`-wrapped | + * + * ## Why the fixture is authored here rather than shared + * + * `FILTER_TEXT_ROWS` is the shared text fixture and this file deliberately does + * NOT use it. That table's rows exist to make CASE and LITERALNESS mistakes + * visible; the mistakes here are about ANCHORING (does a wildcard-free pattern + * match a substring?) and WILDCARD BINDING, which need rows that differ by + * their prefixes and suffixes. Enrolling this file in the shared case-set would + * also flip the conformance gate's `FILTER_TEXT_CASES` cell for a driver on the + * strength of cases that table does not contain. The shared-standard question + * for `$like` is real and is the follow-up the changeset names — it cannot be + * answered by one driver's suite, because `driver-memory` and `driver-mongodb` + * REFUSE these operators today. + * + * ## Reverse verification — predicted before it was run + * + * - Restore `'like': '$contains'` in the spec's `AST_OPERATOR_MAP` → predicted + * RED on the two wire cases below (the wildcard one returns `[]`, the exact + * one returns the substring row too) and GREEN everywhere else, since the + * object spelling does not pass through that map. Measured: exactly that. + * - Delete the `case '$like'` emitter arm → predicted RED on every case in this + * file with an `INVALID_FILTER` refusal rather than a wrong row set. + * Measured: 13 reds, all of them "Unsupported filter operator". + * - Emit `LIKE` instead of `GLOB` on the sqlite arm of `likePatternPredicate` + * → predicted RED on the case-sensitivity case ONLY (SQLite's `LIKE` folds + * ASCII), green elsewhere. Measured: 1 red, that one. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import type { DriverOptions, FilterCondition } from '@objectstack/spec/data'; +import { parseFilterAST } from '@objectstack/spec/data'; +import { SqlDriver } from './sql-driver.js'; + +/** The error a refused filter produced — never a bare `toThrow()`. */ +interface WireBearingError extends Error { + code?: string; + status?: number; +} + +/** + * Rows chosen so every pair differs by the ONE property under test. + * + * `1`/`2`/`3` are the card's own shape: a name that ends with `Industries`, one + * that merely contains it, and the bare word itself. A substring reading and a + * whole-value reading return visibly different sets over them, which is what + * made the original defect invisible — with only row 3 present, the fold and + * the fix agree. + */ +const ROWS = [ + { id: '1', name: 'Acme Industries' }, + { id: '2', name: 'Industries Ltd' }, + { id: '3', name: 'Industries' }, + { id: '4', name: 'ACME INDUSTRIES' }, + { id: '5', name: '100% match' }, + { id: '6', name: '100X match' }, + { id: '7', name: 'a_b' }, + { id: '8', name: 'axb' }, + { id: '9', name: 'a*b' }, +] as const; + +const BYPASS: DriverOptions = { bypassTenantAudit: true }; + +describe('[#7536] SqlDriver — $like / $ilike run the caller\'s pattern', () => { + let driver: SqlDriver; + + beforeAll(async () => { + driver = new SqlDriver({ + client: 'better-sqlite3', + connection: { filename: ':memory:' }, + useNullAsDefault: true, + }); + await driver.initObjects([{ name: 'txt', fields: { name: { type: 'string' } } }]); + for (const row of ROWS) await driver.create('txt', { ...row }, BYPASS); + }); + + afterAll(async () => { + await driver.disconnect(); + }); + + const ids = async (where: FilterCondition): Promise => { + const rows = await driver.find('txt', { where }, BYPASS); + return rows.map((r) => String(r.id)).sort((a, b) => a.localeCompare(b)); + }; + + /** The wire path: an AST array lowered exactly as the protocol door lowers it. */ + const wire = async (ast: unknown): Promise => + ids(parseFilterAST(ast) as FilterCondition); + + const refusalOf = async (where: FilterCondition): Promise => { + const err = await driver + .find('txt', { where }, BYPASS) + .then(() => null, (e: unknown) => e as WireBearingError); + if (!err) throw new Error(`expected a refusal for ${JSON.stringify(where)}, but it compiled`); + return err; + }; + + it('seeded the fixture (the premise)', async () => { + expect(await ids({})).toEqual(['1', '2', '3', '4', '5', '6', '7', '8', '9']); + }); + + // ── The card's repro table, end to end over the wire ────────────────────── + + it('["name","like","%Industries"] returns the rows ENDING WITH Industries', async () => { + // The headline defect: this answered `200` with ZERO rows, because the `%` + // was LIKE-escaped into a literal percent sign by the `$contains` fold. + expect(await wire(['name', 'like', '%Industries'])).toEqual(['1', '3']); + }); + + it('["name","like","Industries"] is an EXACT match, not a substring match', async () => { + // The tell. Under the fold this returned rows 1, 2 and 3 — a substring + // match byte-identical to the `$contains` control below. + expect(await wire(['name', 'like', 'Industries'])).toEqual(['3']); + }); + + it('the $contains control still wraps in %…% — it did NOT move', async () => { + // Half of the guard: `like` had to change WITHOUT `contains` changing. + expect(await wire(['name', 'contains', 'Industries'])).toEqual(['1', '2', '3']); + }); + + it('like and contains no longer answer the same rows', async () => { + // The other half, as a property rather than two literals — this is the + // assertion that catches a future change that moves BOTH spellings. + const like = await wire(['name', 'like', 'Industries']); + const contains = await wire(['name', 'contains', 'Industries']); + expect(like).not.toEqual(contains); + }); + + // ── The pattern language, executed ──────────────────────────────────────── + + it('gives % its "any sequence" meaning at either end', async () => { + expect(await ids({ name: { $like: 'Industries%' } })).toEqual(['2', '3']); + expect(await ids({ name: { $like: '%Industries%' } })).toEqual(['1', '2', '3']); + }); + + it('gives _ its "exactly one character" meaning', async () => { + expect(await ids({ name: { $like: 'a_b' } })).toEqual(['7', '8', '9']); + }); + + it('lets a backslash escape a wildcard back into a literal', async () => { + // Row 7 is the literal `a_b`; rows 8 and 9 must NOT match once the `_` is + // escaped. This is the caller's half of the escaping contract — the + // `$contains` family escapes on the caller's behalf, `$like` does not. + expect(await ids({ name: { $like: 'a\\_b' } })).toEqual(['7']); + expect(await ids({ name: { $like: '100\\% match' } })).toEqual(['5']); + }); + + it('escapes GLOB\'s own metacharacters, which are ORDINARY to LIKE', async () => { + // `*` means nothing to LIKE, so this pattern is a literal — and on the + // SQLite dialects it is compiled to GLOB, where `*` means EVERYTHING. An + // untranslated pattern matches all nine rows here; the translation's + // self-closing `[*]` class is what makes it match one. + expect(await ids({ name: { $like: 'a*b' } })).toEqual(['9']); + }); + + // ── Case: $like is exact, $ilike folds ASCII only ───────────────────────── + + it('$like is case-SENSITIVE (#4706 Q2 = A)', async () => { + // SQLite's LIKE folds ASCII unconditionally, which is why the emitter uses + // GLOB. A driver that regressed to LIKE answers ['1','4'] here. + expect(await ids({ name: { $like: 'Acme%' } })).toEqual(['1']); + }); + + it('$ilike folds ASCII case, on both operands', async () => { + expect(await ids({ name: { $ilike: 'acme%' } })).toEqual(['1', '4']); + expect(await ids({ name: { $ilike: 'ACME%' } })).toEqual(['1', '4']); + }); + + it('lowers a wire `ilike` the same way', async () => { + // `ilike` had NO lowering at all before #7536 — `isFilterAST` refused it, so + // the whole filter was rejected at the protocol door. + expect(await wire(['name', 'ilike', '%industries'])).toEqual(['1', '3', '4']); + }); + + // ── Combinators: the arm must work where filters actually live ──────────── + + it('compiles inside $and / $or', async () => { + expect(await ids({ $or: [{ name: { $like: '%Ltd' } }, { name: { $like: 'a\\_b' } }] })) + .toEqual(['2', '7']); + expect(await ids({ $and: [{ name: { $like: '%Industries' } }, { name: { $like: 'Acme%' } }] })) + .toEqual(['1']); + }); + + it('compiles under $not, and is NULL-safe there', async () => { + // `$not` negates a predicate the driver first makes TOTAL (#5146): SQL's + // `NOT UNKNOWN` is UNKNOWN, which would drop rows whose column is NULL + // where the JS faces keep them. No NULL rows in this fixture, so what is + // pinned is the complement over the rows that exist. + expect(await ids({ $not: { name: { $like: '%Industries' } } })) + .toEqual(['2', '4', '5', '6', '7', '8', '9']); + }); + + // ── Refusals: the shapes that cannot mean one thing on every backend ────── + + it('refuses a non-string pattern in the ADR-0112 envelope', async () => { + const err = await refusalOf({ name: { $like: 42 } } as unknown as FilterCondition); + expect(err.code).toBe('INVALID_FILTER'); + expect(err.status).toBe(400); + expect(err.message).toContain('$like'); + // The refusal must name what to write instead — an author who wanted a + // literal substring search needs `$contains`, not a coerced pattern. + expect(err.message).toContain('$contains'); + }); + + it('refuses an OBJECT pattern rather than compiling "[object Object]"', async () => { + // `String({})` is `[object Object]`, whose `[` OPENS a GLOB character + // class — so coercion here would not merely compare text nobody wrote, it + // would run a PATTERN nobody wrote. + const err = await refusalOf({ name: { $like: {} } } as unknown as FilterCondition); + expect(err.code).toBe('INVALID_FILTER'); + expect(err.status).toBe(400); + }); + + it('refuses a pattern ending in a lone unpaired backslash', async () => { + // No meaning survives every backend: Postgres rejects such a pattern + // outright, and GLOB has no escape character to reject. + const err = await refusalOf({ name: { $like: 'abc\\' } }); + expect(err.code).toBe('INVALID_FILTER'); + expect(err.status).toBe(400); + expect(err.message).toContain('backslash'); + }); + + it('accepts a PAIRED trailing backslash — the refusal is not "contains a backslash"', async () => { + // The gate counts the run length; an even one is a literal backslash and a + // perfectly good pattern. A naïve `endsWith('\\')` test would refuse it. + expect(await ids({ name: { $like: 'a\\\\b' } })).toEqual([]); + }); + + it('accepts an EMPTY pattern, which is not the widening $icontains refuses', async () => { + // `$icontains: ''` is refused because every row contains the empty + // substring. `LIKE ''` matches only the empty string — a narrow, well-formed + // predicate. Copying the sibling's rule here would refuse a legitimate query. + expect(await ids({ name: { $like: '' } })).toEqual([]); + }); +}); diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index 80541dc6ef..92c310da07 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -36,6 +36,12 @@ import { isNowDefaultToken, isRuntimeDefaultToken } from '@objectstack/spec/data // sentence about `$regex` are five sentences that drift apart. This driver // prints `why` VERBATIM. import { RETIRED_FILTER_OPERATORS } from '@objectstack/spec/data'; +// [#7536] `$like`/`$ilike`'s pattern language, defined once in the spec: the +// dangling-escape gate every face refuses on, and the LIKE→GLOB translation the +// SQLite dialects need because GLOB is the only case-exact pattern operator +// SQLite has and it does not speak `%`/`_`. `driver-turso`'s remote transport +// compiles the same operator independently and calls the same two functions. +import { hasDanglingLikeEscape, likePatternToGlobPattern } from '@objectstack/spec/data'; import type { DriverQuery, IDataDriver } from '@objectstack/spec/contracts'; import { StandardErrorCode } from '@objectstack/spec/api'; import { StorageNameMapping } from '@objectstack/spec/system'; @@ -913,6 +919,49 @@ function icontainsComparandError(field: string, value: unknown, path: string): E ); } +/** + * [#7536] `$like` / `$ilike` received a comparand that is not a string. + * + * The declared comparand is `z.string()`, and the coercion the emitter would + * otherwise apply is worse here than at `$icontains`: a `$like` comparand is a + * PATTERN, so `String(value)` does not merely invent text, it invents WILDCARDS + * — `String({})` is `[object Object]`, whose `[` opens a GLOB character class + * on the SQLite dialects. An empty string is deliberately NOT refused (see the + * gate's own note): `LIKE ''` matches only the empty string, which constrains + * plenty. + */ +function likePatternComparandError(field: string, op: string, value: unknown, path: string): Error { + return unsupportedFilterError( + `Operator "${op}" on field "${field}" at ${path} requires a string comparand, received ` + + `${describeFilterOperand(value)} (${safeShapePreview(value)}). "${op}" takes a PATTERN — ` + + `"%" matches any sequence, "_" matches one character, and a backslash escapes either — so ` + + `a non-string comparand cannot be coerced without inventing wildcards the caller never ` + + `wrote. For a literal substring search write "$contains", whose comparand IS text.`, + ); +} + +/** + * [#7536] A `$like` / `$ilike` pattern ending in a lone unpaired backslash. + * + * Refused rather than given a meaning because no meaning survives all five + * backends: Postgres raises `LIKE pattern must not end with escape character` + * at query time, SQLite's GLOB has no escape character at all and would read + * the backslash as an ordinary one, and the JS faces would have to invent a + * third answer. A shape that cannot mean one thing everywhere is refused at the + * door — the direction #5041 / #5234 already settled for comparands this driver + * cannot compile faithfully. `hasDanglingLikeEscape` is the spec's shared test, + * so every face refuses the SAME patterns. + */ +function danglingLikeEscapeError(field: string, op: string, pattern: string, path: string): Error { + return unsupportedFilterError( + `Operator "${op}" on field "${field}" at ${path} has a pattern ending in a lone unpaired ` + + `backslash (${JSON.stringify(pattern)}). A backslash escapes the character after it, so a ` + + `trailing one escapes nothing and the backends disagree about what it means — Postgres ` + + `rejects the pattern outright, SQLite's GLOB has no escape character to reject. Write ` + + `"\\\\\\\\" to match a literal backslash, or drop the trailing one.`, + ); +} + /** * [#5041] The referenced field name when `value` is a Filter Protocol FIELD * REFERENCE (`{ $field: 'other_column' }` — spec `FieldReferenceSchema` in @@ -979,6 +1028,12 @@ function crossFieldComparisonError(field: string, op: string, ref: string, index const SCALAR_COMPARAND_OPERATORS: ReadonlySet = new Set([ '$eq', '$ne', '$gt', '$gte', '$lt', '$lte', '=', '==', '!=', '<>', '>', '>=', '<', '<=', 'like', 'ilike', + // [#7536] The `$`-forms of the two infix spellings already here. They are + // SCALAR rather than {@link TEXT_PATTERN_OPERATORS} for the reason that set's + // note gives: their comparand ARRIVES as the pattern, so nothing wraps or + // escapes it and the question to ask of the value is "can this be bound", + // which is this set's question. + '$like', '$ilike', ]); /** @@ -1005,14 +1060,27 @@ function isBindableComparand(value: unknown): boolean { * first set except a binary buffer is also in the second, but the reason is not * the same reason, and the messages a caller needs differ. * - * `like` / `ilike` are absent on purpose: they arrive already carrying a - * pattern and compile through the scalar bind arm, which already refuses an - * object. + * `like` / `ilike` (and, since #7536, their `$like` / `$ilike` spellings) are + * absent on purpose: they arrive already carrying a pattern and compile + * through the scalar bind arm, which already refuses an object. */ const TEXT_PATTERN_OPERATORS: ReadonlySet = new Set([ '$contains', '$notContains', '$startsWith', '$endsWith', '$icontains', ]); +/** + * [#7536] The two operators whose comparand IS a `LIKE` pattern — the caller's + * own wildcards, neither escaped nor wrapped. + * + * Kept as a named set beside {@link TEXT_PATTERN_OPERATORS} rather than folded + * into it because the two families ask OPPOSITE things of the same string: a + * `$contains` comparand is text that must be made literal (every `%` escaped), + * while a `$like` comparand is a pattern that must be left alone (every `%` is + * the caller's wildcard). Running one through the other's rule is precisely the + * defect #7536 closes. + */ +const LIKE_PATTERN_OPERATORS: ReadonlySet = new Set(['$like', '$ilike']); + /** * [#5234] Operators for which an ARRAY is the legitimate comparand, so it is * each MEMBER that must be individually compilable. @@ -1486,6 +1554,80 @@ function textMatchPredicate( return { sql: `${column} ${keyword} ${comparand} ESCAPE ?`, bindings }; } +/** + * [#7536] The one place a `$like` / `$ilike` PATTERN becomes SQL. + * + * The sibling of {@link textMatchPredicate}, and deliberately a second function + * rather than a flag on it, because the comparand travels in the opposite + * direction through both of that function's jobs: + * + * | | `$contains` family ({@link textMatchPredicate}) | `$like` / `$ilike` (here) | + * |---|---|---| + * | the comparand is | TEXT to match literally | a PATTERN the caller wrote | + * | `%` / `_` in it | escaped, so they are ordinary characters | the caller's WILDCARDS, left alone | + * | wildcards | added by the shape (`%v%`, `v%`, `%v`) | already in the pattern; none added | + * | anchoring | substring / prefix / suffix | the WHOLE value, which is what `LIKE` means | + * + * The dialect matrix is #6518's, unchanged and for its reasons — `$like` is + * case-SENSITIVE (#4706 Q2 = A, the contract its `$contains` sibling answers), + * and `$ilike` folds ASCII and nothing else (Q1 = A): + * + * - **SQLite → `GLOB`**, over {@link likePatternToGlobPattern}'s translation of + * the pattern. `LIKE` there folds ASCII unconditionally and cannot be told + * not to per statement, so a case-exact pattern match has to change operator + * — and changing operator changes the pattern LANGUAGE, which is why the + * translation is a shared spec function instead of an escape call. + * - **Postgres → `LIKE`**, pattern bound verbatim with the bound `ESCAPE`, since + * `LIKE` is already case-exact there. The `$ilike` fold is `translate()` over + * the 26 ASCII letters — deliberately NOT Postgres's own `ILIKE`, which folds + * using the database collation and therefore matches `CAFÉ` against `café`, + * the over-fold #6518 measured and the Q1 = A boundary this must not cross. + * The pattern's `%`, `_` and `\` are not ASCII letters, so `translate()` + * passes them through untouched — the fold cannot corrupt the wildcards. + * - **MySQL → `LIKE` over `CAST(… AS BINARY)`**, byte-wise and so case-exact + * whatever the column collation says; `$ilike` adds + * {@link mysqlAsciiLowerBinary} on both sides. Not executed here for the same + * reason its sibling records: no MySQL server is provisionable in this + * container, so the cell is a declared skip rather than a claimed pass. + * - **`'unknown'` → plain `LIKE` / `LOWER()`**, the only shape that still RUNS + * on a client this driver does not model. Same residue the conformance ledger + * names for the `$contains` family. + */ +function likePatternPredicate( + dialect: SqlDialectName, + field: string, + pattern: string, + fold: boolean, +): { sql: string; bindings: unknown[] } { + if (dialect === 'sqlite') { + // GLOB takes no ESCAPE clause, so this arm binds two values, not three. + const column = fold ? 'lower(??)' : '??'; + const comparand = fold ? 'lower(?)' : '?'; + return { + sql: `${column} GLOB ${comparand}`, + bindings: [field, likePatternToGlobPattern(pattern)], + }; + } + + const bindings = [field, pattern, LIKE_ESCAPE_CHARACTER]; + + if (dialect === 'postgres') { + const asciiLower = (expr: string) => + fold ? `translate(${expr}, '${ASCII_UPPER_LETTERS}', '${ASCII_LOWER_LETTERS}')` : expr; + return { sql: `${asciiLower('??')} LIKE ${asciiLower('?')} ESCAPE ?`, bindings }; + } + + if (dialect === 'mysql') { + const caseExact = (expr: string) => + fold ? mysqlAsciiLowerBinary(expr) : `CAST(${expr} AS BINARY)`; + return { sql: `${caseExact('??')} LIKE ${caseExact('?')} ESCAPE ?`, bindings }; + } + + const column = fold ? 'LOWER(??)' : '??'; + const comparand = fold ? 'LOWER(?)' : '?'; + return { sql: `${column} LIKE ${comparand} ESCAPE ?`, bindings }; +} + /** * [#5134] What a filter node is worth as a boolean, before any SQL is emitted. * @@ -2001,6 +2143,27 @@ function classifyFilterKey(key: string, value: unknown, here: string): FilterVer throw icontainsComparandError(key, value.$icontains, `${here}.$icontains`); } + // [#7536] `$like` / `$ilike` carry a PATTERN, gated on this same walk and for + // the same evaluation-order reason as the three above. + // + // Note what is NOT refused: an EMPTY pattern. `$icontains: ''` is refused + // because every row contains the empty substring, so it constrains nothing — + // but `LIKE ''` matches only the empty string, which is a narrow and perfectly + // well-formed predicate. Copying the sibling's rule here would refuse a + // legitimate query. + if (isFilterNode(value)) { + for (const op of LIKE_PATTERN_OPERATORS) { + if (!Object.prototype.hasOwnProperty.call(value, op)) continue; + const pattern = (value as Record)[op]; + if (typeof pattern !== 'string') { + throw likePatternComparandError(key, op, pattern, `${here}.${op}`); + } + if (hasDanglingLikeEscape(pattern)) { + throw danglingLikeEscapeError(key, op, pattern, `${here}.${op}`); + } + } + } + // A field key always contributes a predicate. return 'clause'; } @@ -8377,6 +8540,28 @@ export class SqlDriver implements IDataDriver { builder[rawMethod](sql, bindings); } + /** + * [#7536] Emit `$like` / `$ilike` — the caller's own pattern, against the + * whole column value. + * + * The sibling of {@link SqlDriver.applyLike} and deliberately not a flag on + * it: see {@link likePatternPredicate} for the table of what the two do + * oppositely. This is the arm the wire could not reach between #5158 and + * #7536, because the spec folded every `like` spelling onto `$contains` + * before a driver ever saw it. + */ + private applyLikePattern( + builder: any, + method: string, + field: string, + pattern: string, + fold: boolean, + ): void { + const rawMethod = method.startsWith('or') ? 'orWhereRaw' : 'whereRaw'; + const { sql, bindings } = likePatternPredicate(this.dialectName, field, pattern, fold); + builder[rawMethod](sql, bindings); + } + /** * Compiles a Filter Protocol condition onto `builder`. * @@ -8631,6 +8816,24 @@ export class SqlDriver implements IDataDriver { case '$endsWith': this.applyLike(builder, method, field, opValue, 'ends'); break; + // [#7536] The pattern pair. NOT `applyLike`: that method escapes the + // comparand and wraps it in the shape's wildcards, which is the + // exact rewrite these two operators exist to avoid. The wildcards + // are the CALLER's, the match is against the WHOLE value, and + // `reduceFilterKey` has already refused a non-string pattern and a + // dangling trailing escape — so `opValue` is a compilable pattern + // by the time it reaches here. + // + // `opValue` rather than `coerced`: `coerceFilterValue` canonicalises + // a comparand against the column's TYPE (dates, booleans, numbers), + // and a pattern is not a value of that type — coercing it would + // rewrite the pattern text. + case '$like': + this.applyLikePattern(builder, method, field, opValue as string, false); + break; + case '$ilike': + this.applyLikePattern(builder, method, field, opValue as string, true); + break; case '$between': { const arr = Array.isArray(coerced) ? coerced : []; if (arr.length !== 2) { @@ -8671,7 +8874,7 @@ export class SqlDriver implements IDataDriver { throw unsupportedFilterError( `Unsupported filter operator "${op}" on field "${field}". Supported operators: ` + `$eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $between, $contains, $notContains, ` + - `$startsWith, $endsWith, $icontains, $null, $exists.`, + `$startsWith, $endsWith, $icontains, $like, $ilike, $null, $exists.`, ); } } diff --git a/packages/drivers/driver-turso/src/remote-transport.ts b/packages/drivers/driver-turso/src/remote-transport.ts index bcb4a81828..cb366cb68c 100644 --- a/packages/drivers/driver-turso/src/remote-transport.ts +++ b/packages/drivers/driver-turso/src/remote-transport.ts @@ -15,6 +15,11 @@ import type { Client, InStatement, ResultSet } from '@libsql/client'; import { StandardErrorCode } from '@objectstack/spec/api'; import { FILTER_OPERATORS, LOGICAL_OPERATORS, RETIRED_FILTER_OPERATORS } from '@objectstack/spec/data'; +// [#7536] `$like`/`$ilike`'s pattern language, from the spec's one definition — +// the dangling-escape gate and the LIKE→GLOB translation. Shared with +// `SqlDriver`'s local emitter so this transport and its local twin cannot fork +// on what a pattern means (the fork `turso-local-remote-*` suites exist to catch). +import { hasDanglingLikeEscape, likePatternToGlobPattern } from '@objectstack/spec/data'; // The DECLARED aggregate vocabulary (#5907) — read from the spec so this // transport's "the protocol has no such function" refusal cannot drift from what // `AggregationNodeSchema.function` admits, nor from the local driver's twin. @@ -68,6 +73,13 @@ const SUPPORTED_FILTER_OPERATORS = [ '$startsWith', '$endsWith', '$icontains', + // [#7536] The pattern pair, declared by `StringOperatorSchema` and compiled + // here. They must be listed for the same reason `$icontains` is: this + // transport's LOCAL twin (`TursoDriver extends SqlDriver`) compiles them, and + // a transport that refused what its own local mode answers is the local/remote + // fork `turso-local-remote-*` parity suites exist to prevent. + '$like', + '$ilike', '$null', '$exists', ] as const; @@ -110,6 +122,12 @@ const NODE_COMBINATORS: ReadonlySet = new Set(LOGICAL_OPERATORS) const MISPLACED_FIELD_OPERATORS: ReadonlySet = new Set([ ...FILTER_OPERATORS, '$icontains', + // [#7536] Staged out of `FILTER_OPERATORS` exactly like `$icontains`, and + // compiled here exactly like it — so a node-position `$like` really is a + // MISPLACED field operator, and must get that repair rather than "names + // nothing this protocol declares". + '$like', + '$ilike', ]); /** @@ -2012,6 +2030,24 @@ export class RemoteTransport { case '$endsWith': this.pushLike(clauses, args, column, this.serializeComparand(object, key, op, opValue), 'ends'); break; + // [#7536] `$like` / `$ilike` — the caller's OWN pattern, matched + // against the whole value. Deliberately NOT `pushLike`: that method + // escapes the comparand and wraps it in a shape's wildcards, which + // is the rewrite these two operators exist to avoid. They go through + // {@link RemoteTransport.pushLikePattern} instead, which shares the + // spec's LIKE→GLOB translation with the local twin so both + // transports answer one pattern language. + case '$like': + case '$ilike': { + if (typeof opValue !== 'string') { + throw this.likePatternComparand(object, key, op, opValue); + } + if (hasDanglingLikeEscape(opValue)) { + throw this.danglingLikeEscape(object, key, op, opValue); + } + this.pushLikePattern(clauses, args, column, opValue, op === '$ilike'); + break; + } // ── Existence ──────────────────────────────────────────────── // `{ $null: true }` → IS NULL, `{ $null: false }` → IS NOT NULL; // `$exists` is its inverse. Both compare the presence of a value, @@ -2190,6 +2226,77 @@ export class RemoteTransport { args.push(pattern); } + /** + * [#7536] Append one parameterized `$like` / `$ilike` predicate. + * + * The sibling of {@link RemoteTransport.pushLike}, and a separate method for + * the reason that one's docblock makes unavoidable: `pushLike` ESCAPES the + * comparand's metacharacters and WRAPS it in the shape's wildcards, because + * its operators take text. `$like` takes a pattern, so both of those steps + * are exactly wrong here — the `%` and `_` in it are the caller's wildcards, + * and the match is against the whole value, not a substring of it. + * + * `GLOB` for the same reason the sibling emits it (#6518): libSQL is SQLite, + * SQLite's `LIKE` folds ASCII case unconditionally, and `$like` is + * case-SENSITIVE by contract. Because GLOB speaks a different pattern + * language, the comparand is TRANSLATED rather than escaped — + * `likePatternToGlobPattern` is the spec's one definition of that + * translation, shared with `SqlDriver`'s local emitter so the two transports + * cannot answer one pattern two ways. + * + * `fold` wraps BOTH operands in `lower()`, the `$ilike` fold: folding only the + * pattern would compare a folded needle against a raw column and match just + * the rows that were already lower-case. `lower()` on SQLite folds ASCII only, + * which IS the contract (#4706 Q1 = A) rather than a limitation. + */ + private pushLikePattern( + clauses: string[], + args: any[], + column: string, + pattern: string, + fold: boolean, + ): void { + const lhs = fold ? `lower(${column})` : column; + const rhs = fold ? 'lower(?)' : '?'; + clauses.push(`${lhs} GLOB ${rhs}`); + args.push(likePatternToGlobPattern(pattern)); + } + + /** + * [#7536] The error for a `$like` / `$ilike` comparand that is not a string. + * + * The remote twin of `driver-sql`'s `likePatternComparandError`. Coercion is + * worse here than for the text operators: `String({})` is `[object Object]`, + * whose `[` OPENS a GLOB character class, so the query would run a pattern the + * caller never wrote instead of merely comparing text they never wrote. + */ + private likePatternComparand(object: string, field: string, op: string, value: unknown): Error { + return invalidFilterError( + `[RemoteTransport] Operator "${op}" on '${object}.${field}' requires a string comparand. ` + + `Received ${typeof value} (${preview(value)}). "${op}" takes a PATTERN — "%" matches any ` + + `sequence, "_" matches one character, a backslash escapes either — so a non-string ` + + `comparand cannot be coerced without inventing wildcards. For a literal substring search ` + + `write "$contains", whose comparand IS text. @objectstack/spec StringOperatorSchema ` + + `declares ${op} as a string.`, + ); + } + + /** + * [#7536] The error for a `$like` / `$ilike` pattern ending in a lone + * unpaired backslash — refused because no meaning survives every backend + * (Postgres rejects such a pattern outright; GLOB has no escape character at + * all). `hasDanglingLikeEscape` is the spec's shared test, so this transport + * refuses the same patterns as its local twin and the JS faces. + */ + private danglingLikeEscape(object: string, field: string, op: string, pattern: string): Error { + return invalidFilterError( + `[RemoteTransport] Operator "${op}" on '${object}.${field}' has a pattern ending in a lone ` + + `unpaired backslash (${JSON.stringify(pattern)}). A backslash escapes the character after ` + + `it, so a trailing one escapes nothing and the backends disagree about what it means. ` + + `Write "\\\\\\\\" to match a literal backslash, or drop the trailing one.`, + ); + } + /** * [#5298] Wrap a negative-polarity value test so a row whose column has no * value SATISFIES it: `(col IS NULL OR )`. diff --git a/packages/drivers/driver-turso/src/turso-local-remote-like-parity.test.ts b/packages/drivers/driver-turso/src/turso-local-remote-like-parity.test.ts new file mode 100644 index 0000000000..c80a264969 --- /dev/null +++ b/packages/drivers/driver-turso/src/turso-local-remote-like-parity.test.ts @@ -0,0 +1,150 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#7536] `$like` / `$ilike` answer the SAME rows on BOTH turso transports. + * + * ## Why this driver needs its own parity suite for these two operators + * + * `TursoDriver` is the one driver with TWO filter compilers. Local mode extends + * `SqlDriver` and inherits its emitter; remote mode compiles SQL itself in + * `RemoteTransport.buildWhereSQL`, because it speaks a wire protocol rather + * than knex. So every operator has to be implemented twice here, and "twice" + * is where a pattern language forks: the two could easily agree about + * `%Industries` and disagree about `a\\_b` or `a*b`, which no single-transport + * suite would notice. + * + * #7536 narrows that risk deliberately — both sides call the SPEC's + * `likePatternToGlobPattern`, so the translation itself is one function rather + * than two. What is still per-transport, and therefore what these cases + * actually pin, is everything around it: which operand gets the `lower()` fold, + * whether the pattern is bound or interpolated, and whether the comparand gate + * fires with the same verdict on both sides. + * + * The GLOB detour is #6518's and applies with full force here: libSQL is + * SQLite, SQLite's `LIKE` folds ASCII unconditionally, and `$like` is + * case-SENSITIVE by contract — so a transport that reached for `LIKE` would + * over-match, and only the case rows below would catch it. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import type { DriverQuery } from '@objectstack/spec/contracts'; +import { TursoDriver } from './turso-driver.js'; +import { asLibsqlClient, makeLibsqlSqliteStub, type LibsqlSqliteStub } from './libsql-sqlite-stub.testkit.js'; + +const OBJECT = { + name: 'like_parity', + fields: { name: { type: 'string' } }, +}; + +/** + * The same shape `sql-driver-like-pattern.test.ts` uses: rows that differ by + * PREFIX and SUFFIX, so a whole-value match and a substring match return + * visibly different sets. With only the bare word present the fold and the fix + * agree, which is how the original defect stayed invisible. + */ +const ROWS = [ + { id: '1', name: 'Acme Industries' }, + { id: '2', name: 'Industries Ltd' }, + { id: '3', name: 'Industries' }, + { id: '4', name: 'ACME INDUSTRIES' }, + { id: '5', name: '100% match' }, + { id: '6', name: '100X match' }, + { id: '7', name: 'a_b' }, + { id: '8', name: 'axb' }, + { id: '9', name: 'a*b' }, +] as const; + +interface WireBearingError extends Error { + code?: string; + status?: number; +} + +const ids = (rows: Array>): string[] => + rows.map((r) => String(r.id)).sort((x, y) => x.localeCompare(y)); + +describe('[#7536] TursoDriver LOCAL and REMOTE run one pattern language', () => { + let local: TursoDriver; + let remote: TursoDriver; + let stub: LibsqlSqliteStub; + + beforeAll(async () => { + local = new TursoDriver({ url: ':memory:' }); + expect(local.transportMode).toBe('local'); + await local.initObjects([OBJECT]); + for (const row of ROWS) await local.create(OBJECT.name, { ...row }, { bypassTenantAudit: true }); + + stub = makeLibsqlSqliteStub(); + remote = new TursoDriver({ url: 'libsql://like-parity.turso.io', client: asLibsqlClient(stub) }); + await remote.connect(); + expect(remote.transportMode).toBe('remote'); + await remote.syncSchema(OBJECT.name, OBJECT); + for (const row of ROWS) await remote.create(OBJECT.name, { ...row }); + }); + + afterAll(async () => { + await local.disconnect(); + await remote.disconnect(); + stub.close(); + }); + + it('both transports hold the same nine rows (the premise)', async () => { + const all = ['1', '2', '3', '4', '5', '6', '7', '8', '9']; + expect(ids(await local.find(OBJECT.name, {}))).toEqual(all); + expect(ids(await remote.find(OBJECT.name, {}))).toEqual(all); + }); + + /** Every case asserts the two transports agree BEFORE asserting the answer. */ + const CASES: Array<[string, Record, string[]]> = [ + // The card's own table. + ['a wildcard pattern binds the caller\'s %', { name: { $like: '%Industries' } }, ['1', '3']], + ['a wildcard-free pattern is EXACT, not a substring', { name: { $like: 'Industries' } }, ['3']], + ['% at the end', { name: { $like: 'Industries%' } }, ['2', '3']], + // The pattern language. + ['_ matches exactly one character', { name: { $like: 'a_b' } }, ['7', '8', '9']], + ['a backslash escapes _ back to a literal', { name: { $like: 'a\\_b' } }, ['7']], + ['a backslash escapes % back to a literal', { name: { $like: '100\\% match' } }, ['5']], + // GLOB's own metacharacters are ORDINARY to LIKE. Untranslated, `a*b` + // matches every row on a SQLite engine — the widening the shared + // translation's self-closing `[*]` class prevents, on both transports. + ['* is a literal, not a GLOB wildcard', { name: { $like: 'a*b' } }, ['9']], + // Case. + ['$like is case-SENSITIVE', { name: { $like: 'Acme%' } }, ['1']], + ['$ilike folds ASCII case', { name: { $ilike: 'acme%' } }, ['1', '4']], + ['$ilike folds the other direction too', { name: { $ilike: 'ACME%' } }, ['1', '4']], + // The control: `$contains` must NOT have moved. + ['the $contains control still wraps in %…%', { name: { $contains: 'Industries' } }, ['1', '2', '3']], + ]; + + for (const [name, where, expected] of CASES) { + it(name, async () => { + const localIds = ids(await local.find(OBJECT.name, { where } as DriverQuery)); + const remoteIds = ids(await remote.find(OBJECT.name, { where } as DriverQuery)); + // The difference IS the assertion — reported first, so a fork between the + // two emitters reads as a fork rather than as two unrelated wrong answers. + expect(remoteIds, 'remote disagrees with local').toEqual(localIds); + expect(localIds).toEqual(expected); + }); + } + + // ── Refusals must match too, envelope included ──────────────────────────── + + const REFUSED: Array<[string, Record, string]> = [ + ['a non-string pattern', { name: { $like: 42 } }, '$like'], + ['an object pattern', { name: { $like: {} } }, '$like'], + ['a dangling trailing backslash', { name: { $like: 'abc\\' } }, 'backslash'], + ]; + + for (const [name, where, mention] of REFUSED) { + it(`refuses ${name} on BOTH transports, in the ADR-0112 envelope`, async () => { + for (const [face, driver] of [['local', local], ['remote', remote]] as const) { + const err = await driver + .find(OBJECT.name, { where } as DriverQuery) + .then(() => null, (e: unknown) => e as WireBearingError); + expect(err, `${face} compiled a filter that must be refused`).not.toBeNull(); + expect(err!.code, face).toBe('INVALID_FILTER'); + expect(err!.status, face).toBe(400); + expect(err!.message, `${face} — ${mention}`).toContain(mention); + } + }); + } +}); diff --git a/packages/formula/src/matches-filter-like.test.ts b/packages/formula/src/matches-filter-like.test.ts new file mode 100644 index 0000000000..3fc72ef159 --- /dev/null +++ b/packages/formula/src/matches-filter-like.test.ts @@ -0,0 +1,92 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#7536] `$like` / `$ilike` on the write-side evaluator. + * + * ## Why this face gets ARMS while driver-memory gets a refusal + * + * The two are not inconsistent, they are answering different questions. A + * DRIVER compiles a read: it can refuse a filter it cannot express, and the + * caller sees a 400. This evaluator answers an RLS write-side `check` per + * record, and its documented posture is that an operator it does not know + * returns `false` — fail-closed, deny the write. That posture is right for a + * spelling the protocol does not have, and wrong for one it DOES: a declared + * `$like` hitting the silent default would deny every write while the read + * scope's SQL happily matched rows. One predicate, two answers, across the + * write gate and the read gate — the #3948 shape, and exactly the defect the + * #6993 census measured for `$icontains`. + * + * So the rule this file enforces is the one `matches-filter.ts`' header states: + * every operator an author may WRITE has an arm here. The test is the + * declaration surface (`StringOperatorSchema`), not the `FILTER_OPERATORS` + * allowlist — `$like` is staged out of that array on purpose, and staging must + * not be a way to reintroduce the silent `false`. + * + * The semantics are the spec's shared `matchesLikePattern`, which is what + * `driver-sql` compiles to `LIKE` / `GLOB` — so a `check` evaluated here and + * the same predicate compiled to SQL cannot disagree. + */ + +import { describe, it, expect } from 'vitest'; +import { matchesFilterCondition } from './matches-filter.js'; + +const row = (name: unknown) => ({ name }); + +describe('[#7536] matchesFilterCondition — $like / $ilike', () => { + it('matches the WHOLE value, so a wildcard-free pattern is exact', () => { + expect(matchesFilterCondition(row('Industries'), { name: { $like: 'Industries' } })).toBe(true); + // The defect's signature: a substring must NOT match. + expect(matchesFilterCondition(row('Acme Industries'), { name: { $like: 'Industries' } })).toBe(false); + }); + + it('binds the caller\'s wildcards', () => { + expect(matchesFilterCondition(row('Acme Industries'), { name: { $like: '%Industries' } })).toBe(true); + expect(matchesFilterCondition(row('Industries Ltd'), { name: { $like: '%Industries' } })).toBe(false); + expect(matchesFilterCondition(row('axb'), { name: { $like: 'a_b' } })).toBe(true); + expect(matchesFilterCondition(row('ab'), { name: { $like: 'a_b' } })).toBe(false); + }); + + it('honours a backslash escape', () => { + expect(matchesFilterCondition(row('a_b'), { name: { $like: 'a\\_b' } })).toBe(true); + expect(matchesFilterCondition(row('axb'), { name: { $like: 'a\\_b' } })).toBe(false); + }); + + it('is case-SENSITIVE for $like and ASCII-folded for $ilike', () => { + expect(matchesFilterCondition(row('ACME Corp'), { name: { $like: 'acme%' } })).toBe(false); + expect(matchesFilterCondition(row('ACME Corp'), { name: { $ilike: 'acme%' } })).toBe(true); + // The #4706 Q1 = A boundary — the fold is ASCII and nothing else. A + // `toLowerCase()` implementation answers `true` here and diverges from the + // SQL read side, which is the whole reason the fold lives in the spec. + expect(matchesFilterCondition(row('CAFÉ'), { name: { $ilike: 'café' } })).toBe(false); + }); + + it('does NOT agree with $contains — the two operators stayed distinct', () => { + const record = row('Acme Industries'); + expect(matchesFilterCondition(record, { name: { $contains: 'Industries' } })).toBe(true); + expect(matchesFilterCondition(record, { name: { $like: 'Industries' } })).toBe(false); + }); + + it('answers FALSE for a non-string value rather than throwing', () => { + // This evaluator is TOTAL by contract: `plugin-security`'s explain engine + // calls it per record, so a throw would turn a per-record verdict into an + // aborted operation. + expect(matchesFilterCondition(row(42), { name: { $like: '%4%' } })).toBe(false); + expect(matchesFilterCondition(row(null), { name: { $like: '%' } })).toBe(false); + }); + + it('answers FALSE for a malformed pattern rather than throwing', () => { + // A dangling trailing escape is refused LOUDLY by the driver faces, where a + // caller can act on it. Here the fail-closed answer is the safe one, and + // totality is the contract. + expect(matchesFilterCondition(row('abc'), { name: { $like: 'abc\\' } })).toBe(false); + }); + + it('is reached through combinators too', () => { + expect(matchesFilterCondition(row('Acme Industries'), { + $or: [{ name: { $like: 'nope' } }, { name: { $like: '%Industries' } }], + })).toBe(true); + expect(matchesFilterCondition(row('Acme Industries'), { + $not: { name: { $like: '%Industries' } }, + })).toBe(false); + }); +}); diff --git a/packages/formula/src/matches-filter.ts b/packages/formula/src/matches-filter.ts index e8e6259318..6ade76cfaf 100644 --- a/packages/formula/src/matches-filter.ts +++ b/packages/formula/src/matches-filter.ts @@ -39,7 +39,13 @@ * was that a spec-DECLARED operator (`$icontains`) got the silent `false`. * Every operator in `FILTER_OPERATORS` now has an arm in {@link evalOp}, so * the silent answer is reachable only for a name the protocol does not - * declare or has retired. + * declare or has retired. [#7536] That claim is maintained rather than + * merely inherited: `$like` / `$ilike` are DECLARED by + * `StringOperatorSchema` while deliberately staged out of + * `FILTER_OPERATORS`, and they got arms here in the PR that declared them — + * the test is "can an author write it", not "is it in the allowlist", and by + * that test a silent `false` for `$like` would have been the same defect + * under a new name. * * What stays open, deliberately and on the record: a RETIRED spelling * (`$regex` / `$options`) still gets the silent `false` here while the other five @@ -65,6 +71,11 @@ import type { FilterCondition } from '@objectstack/spec/data'; // and the same predicate compiled to SQL by `read-scope-sql.ts` fold the same // domain. import { nextUtcCalendarDay, utcInstantMs, asciiCaseInsensitiveContains } from '@objectstack/spec/data'; +// [#7536] `$like`/`$ilike`'s pattern language, likewise defined once in the +// spec: this face evaluates the pattern in JS, `driver-sql` compiles the same +// one to `LIKE`/`GLOB`, and a translation written twice would agree on the day +// it was typed and never again. +import { matchesLikePattern } from '@objectstack/spec/data'; import { StandardErrorCode } from '@objectstack/spec/api'; /** @@ -233,6 +244,36 @@ function evalOp(actual: unknown, op: string, raw: unknown, record: Record { + // ── The card's repro table, as pins ─────────────────────────────────────── + + it('keeps a wildcard pattern INTACT — the `%` is the caller\'s, not a literal', () => { + // Row 1 of the table. Under the fold this lowered to + // `{ name: { $contains: '%Industries' } }`, which driver-sql compiled to + // `LIKE '%\%Industries%'` — a search for a literal percent sign, hence the + // measured 0 rows. + expect(parseFilterAST(['name', 'like', '%Industries'])).toEqual({ + name: { $like: '%Industries' }, + }); + }); + + it('lowers a WILDCARD-FREE `like` to an exact pattern, NOT a substring search', () => { + // Row 2, and the load-bearing one: this is the assertion that goes red if + // anyone restores the `'like': '$contains'` map entry. + expect(parseFilterAST(['name', 'like', 'Industries'])).toEqual({ + name: { $like: 'Industries' }, + }); + }); + + it('does NOT produce the same lowering as the `$contains` control', () => { + // The byte-identical response QA measured, restated as a property. Stated + // as an inequality on purpose: a future refactor could change BOTH spellings + // and the two assertions above would still pass while this one catches the + // collapse. + const like = parseFilterAST(['name', 'like', 'Industries']); + const contains = parseFilterAST(['name', 'contains', 'Industries']); + expect(like).not.toEqual(contains); + }); + + it('leaves the `$contains` control lowering exactly as it was', () => { + // The other half of the same guard: `like` must move WITHOUT `contains` + // moving. This is what goes red if someone "fixes" the divergence by + // changing the control instead. + expect(parseFilterAST(['name', 'contains', 'Industries'])).toEqual({ + name: { $contains: 'Industries' }, + }); + }); + + // ── `ilike`, which had no lowering at all before this ───────────────────── + + it('accepts `ilike` and lowers it to its own operator', () => { + // Before #7536 `ilike` was absent from `AST_OPERATOR_MAP` entirely, so + // `isFilterAST` REFUSED it and the whole filter was rejected at the door. + expect(isFilterAST(['name', 'ilike', '%industries'])).toBe(true); + expect(parseFilterAST(['name', 'ilike', '%industries'])).toEqual({ + name: { $ilike: '%industries' }, + }); + }); + + it('carries both spellings in the AST vocabulary', () => { + expect(VALID_AST_OPERATORS.has('like')).toBe(true); + expect(VALID_AST_OPERATORS.has('ilike')).toBe(true); + }); + + it('folds the operator case-insensitively, like every other spelling', () => { + expect(parseFilterAST(['name', 'LIKE', '%x'])).toEqual({ name: { $like: '%x' } }); + expect(parseFilterAST(['name', 'ILike', '%x'])).toEqual({ name: { $ilike: '%x' } }); + }); + + // ── The nested / negated positions the AST admits ───────────────────────── + + it('lowers a `like` inside an `and` node', () => { + expect(parseFilterAST(['and', ['name', 'like', '%Industries'], ['stage', '=', 'won']])).toEqual({ + $and: [{ name: { $like: '%Industries' } }, { stage: 'won' }], + }); + }); + + it('lowers a `like` inside an `or` node beside its `contains` control', () => { + // Both operators in ONE filter, which is the shape that makes a fold + // visible at a glance: under the defect these two branches were identical. + expect(parseFilterAST(['or', ['name', 'like', 'Acme'], ['name', 'contains', 'Acme']])).toEqual({ + $or: [{ name: { $like: 'Acme' } }, { name: { $contains: 'Acme' } }], + }); + }); + + it('lowers a `like` in the legacy flat-array position', () => { + expect(parseFilterAST([['name', 'like', 'a%'], ['stage', '=', 'won']])).toEqual({ + $and: [{ name: { $like: 'a%' } }, { stage: 'won' }], + }); + }); + + // The AST comparison grammar has no negation operator — `not_contains` is a + // distinct operator rather than a modifier, and `$not` is reachable only from + // the object spelling. So the negated position is exercised where it exists: + // as a `FilterCondition`, which needs no lowering. + + // ── `canonicalAstOperator` still answers, now by construction ───────────── + + it('canonicalises both spellings to themselves, without a hand-written exemption', () => { + // This function always answered correctly — the hand-written exemption it + // used to carry is what the defect's own comment documented. It now falls + // out of the generic round-trip (`like` → `$like` → `like`), which is why + // the exemption is gone. Pinned so the answer cannot regress with it. + expect(canonicalAstOperator('like')).toBe('like'); + expect(canonicalAstOperator('ilike')).toBe('ilike'); + expect(canonicalAstOperator('LIKE')).toBe('like'); + }); + + it('does not canonicalise `like` onto `contains`', () => { + expect(canonicalAstOperator('like')).not.toBe(canonicalAstOperator('contains')); + }); +}); + +describe('[#7536] the `$like` pattern language', () => { + // One definition, shared by every face — so these cases are the contract the + // SQL emitters and the JS evaluator are BOTH held to. + + it('matches the whole value, so a wildcard-free pattern is EXACT', () => { + expect(matchesLikePattern('Industries', 'Industries')).toBe(true); + // The defect's signature, as a behaviour: a substring must NOT match. + expect(matchesLikePattern('Acme Industries', 'Industries')).toBe(false); + }); + + it('gives `%` its "any sequence" meaning, including empty', () => { + expect(matchesLikePattern('Acme Industries', '%Industries')).toBe(true); + expect(matchesLikePattern('Industries', '%Industries')).toBe(true); + expect(matchesLikePattern('Industries Ltd', '%Industries')).toBe(false); + expect(matchesLikePattern('Acme Industries Ltd', '%Industries%')).toBe(true); + }); + + it('gives `_` its "exactly one character" meaning', () => { + expect(matchesLikePattern('axb', 'a_b')).toBe(true); + expect(matchesLikePattern('ab', 'a_b')).toBe(false); + expect(matchesLikePattern('axyb', 'a_b')).toBe(false); + }); + + it('lets a backslash escape a wildcard back into a literal', () => { + expect(matchesLikePattern('100% match', '100\\% match')).toBe(true); + expect(matchesLikePattern('100X match', '100\\% match')).toBe(false); + expect(matchesLikePattern('a_b', 'a\\_b')).toBe(true); + expect(matchesLikePattern('axb', 'a\\_b')).toBe(false); + expect(matchesLikePattern('a\\b', 'a\\\\b')).toBe(true); + }); + + it('treats regex metacharacters as LITERAL — the pattern language is LIKE, not regex', () => { + // The `$regex` defect (#4706) restated one operator over: `.` is an + // ordinary character to LIKE, and a translation that forgot to escape it + // would match `axb` here. + expect(matchesLikePattern('a.b', 'a.b')).toBe(true); + expect(matchesLikePattern('axb', 'a.b')).toBe(false); + expect(matchesLikePattern('a+b', 'a+b')).toBe(true); + }); + + it('is case-SENSITIVE for `$like` and ASCII-folded for `$ilike`', () => { + expect(matchesLikePattern('ACME Corp', 'acme%')).toBe(false); + expect(matchesLikePattern('ACME Corp', 'acme%', true)).toBe(true); + // The #4706 Q1 = A boundary: the fold is ASCII and nothing else. + expect(matchesLikePattern('CAFÉ', 'café', true)).toBe(false); + expect(matchesLikePattern('CAFE', 'cafe', true)).toBe(true); + }); + + it('matches across newlines, because SQL `%` has no line concept', () => { + expect(matchesLikePattern('a\nb', 'a%b')).toBe(true); + expect(matchesLikePattern('a\nb', 'a_b')).toBe(true); + }); + + it('anchors the source, so a pattern cannot match a substring by accident', () => { + expect(likePatternToRegexSource('abc')).toBe('^abc$'); + }); + + it('refuses a pattern ending in a lone unpaired backslash', () => { + expect(hasDanglingLikeEscape('abc\\')).toBe(true); + expect(hasDanglingLikeEscape('abc\\\\')).toBe(false); + expect(hasDanglingLikeEscape('a\\\\\\')).toBe(true); + expect(hasDanglingLikeEscape('abc')).toBe(false); + expect(() => likePatternToRegexSource('abc\\')).toThrow(/unpaired backslash/); + expect(() => likePatternToGlobPattern('abc\\')).toThrow(/unpaired backslash/); + }); +}); + +describe('[#7536] the LIKE → GLOB translation the SQLite dialects need', () => { + // SQLite's `LIKE` folds ASCII case and cannot be told not to per statement, + // so a case-exact pattern match has to use `GLOB` — which speaks a DIFFERENT + // pattern language. These cases are what stops the two from being confused. + + it('translates the wildcards', () => { + expect(likePatternToGlobPattern('%Industries')).toBe('*Industries'); + expect(likePatternToGlobPattern('a_b')).toBe('a?b'); + expect(likePatternToGlobPattern('%a_b%')).toBe('*a?b*'); + }); + + it('escapes GLOB\'s OWN metacharacters, which are ORDINARY to LIKE', () => { + // The direction a hand-written escape forgets. An unescaped `*` in a GLOB + // pattern is the same filter bypass an unescaped `%` is under LIKE (#5567). + expect(likePatternToGlobPattern('a*b')).toBe('a[*]b'); + expect(likePatternToGlobPattern('a?b')).toBe('a[?]b'); + expect(likePatternToGlobPattern('a[b')).toBe('a[[]b'); + // `]` needs no escape: every `[` became a class that closes itself, so no + // unclosed class survives for a later `]` to terminate. + expect(likePatternToGlobPattern('a]b')).toBe('a]b'); + }); + + it('turns an ESCAPED LIKE wildcard into a literal GLOB character', () => { + expect(likePatternToGlobPattern('100\\%')).toBe('100%'); + expect(likePatternToGlobPattern('a\\_b')).toBe('a_b'); + // …and re-escapes it when the literal is a GLOB metacharacter. + expect(likePatternToGlobPattern('a\\*b')).toBe('a[*]b'); + }); + + it('translates the card\'s own patterns to the GLOB the SQLite dialects run', () => { + // The end-to-end row counts these produce are pinned against a real SQLite + // engine in `sql-driver-like-pattern.test.ts`; what belongs HERE is the + // exact string, so a change to the translation is visible in this package's + // own diff rather than only as a driver test failing later. + expect(likePatternToGlobPattern('%Industries')).toBe('*Industries'); + expect(likePatternToGlobPattern('Industries')).toBe('Industries'); + expect(likePatternToGlobPattern('100\\%%')).toBe('100%*'); + }); + + it('never emits a GLOB wildcard the LIKE pattern did not ask for', () => { + // The property a broken translation violates, stated over every character + // that means something to either language: the only `*` and `?` in the + // output are the ones `%` and `_` put there. Everything else is inside a + // self-closing class, so it cannot widen the match. + for (const literal of ['*', '?', '[', ']', '%', '_', '\\']) { + const pattern = `a${literal === '%' || literal === '_' || literal === '\\' ? '\\' : ''}${literal}b`; + const glob = likePatternToGlobPattern(pattern); + // Strip the escaped classes; nothing wildcard-ish may remain. + expect(glob.replace(/\[.\]/g, '')).not.toMatch(/[*?]/); + } + }); +}); diff --git a/packages/spec/src/data/filter-operator-vocabulary.test.ts b/packages/spec/src/data/filter-operator-vocabulary.test.ts index e5228d97ff..4dc0fc2584 100644 --- a/packages/spec/src/data/filter-operator-vocabulary.test.ts +++ b/packages/spec/src/data/filter-operator-vocabulary.test.ts @@ -13,10 +13,13 @@ * gate and `service-analytics`' coverage test DERIVE from. An entry here is a * claim that backends implement the operator. * - * The two surfaces AGREE today, and the staging that made them disagree is - * over: #6520 added `$icontains` to `FILTER_OPERATORS` in the same PR that gave - * every JS evaluation face an arm, which is the only order in which that name - * could be added at all. Measured on the branch that added it EARLY (#5701): + * The two surfaces disagree by exactly {@link STAGED_AHEAD_OF_BACKENDS}, and + * they disagree DELIBERATELY. `$icontains`' staging is over — #6520 added it to + * `FILTER_OPERATORS` in the same PR that gave every JS evaluation face an arm, + * which is the only order in which that name could be added at all — and + * #7536's `$like`/`$ilike` are staged the same way, answered by the SQL family + * and `formula` while the remaining faces refuse them loudly. Measured on the + * branch that added `$icontains` EARLY (#5701): * driver-memory's gate stopped refusing it and * `match({ name: 'zzz' }, { name: { $icontains: 'acme' } })` returned `true` — * the predicate silently dropped, every row matched. That is the widening #3948 @@ -39,6 +42,25 @@ import { const declaredKeys = () => Object.keys(FieldOperatorsSchema.shape).sort(); +/** + * [#7536] The operators declared ahead of their backends, and the issue that + * clears each. Adding a name here is the deliberate act the assertion below + * demands — it is not a way to silence the check, because the equality still + * fails the moment a name is declared and NOT recorded, or recorded and no + * longer staged. + * + * `$like` / `$ilike`: declared by `StringOperatorSchema` and answered by the + * SQL family (driver-sql, driver-sqlite-wasm, driver-turso on both transports) + * and by `@objectstack/formula`. `driver-memory`, `driver-mongodb`, objectql + * `having` and `service-analytics` refuse them loudly in the ADR-0112 + * envelope, which is what keeping them OUT of `FILTER_OPERATORS` buys: + * driver-memory's shape gate derives from that array, so membership would flip + * its refusal into a DROPPED predicate (measured for `$icontains` in #5701 — + * `match()` returned `true` for a non-matching record). Cleared by giving the + * remaining faces arms in one PR, the #6520 direction. + */ +const STAGED_AHEAD_OF_BACKENDS = ['$ilike', '$like']; + describe('the declaration surface and the enforcement surface', () => { it('differ by EXACTLY the operators staged ahead of their backends', () => { const declared = new Set(declaredKeys()); @@ -57,7 +79,7 @@ describe('the declaration surface and the enforcement surface', () => { + 'in ONE PR — spec word list, driver-memory (query path, reference matcher, analytics ' + 'face), driver-mongodb, service-analytics (3 compilers), objectql `having`, formula — ' + 'then empty this list. #6520 is the worked example of the clearing direction.', - ).toEqual([]); + ).toEqual(STAGED_AHEAD_OF_BACKENDS); }); it('has no operator enforced that is not declared', () => { diff --git a/packages/spec/src/data/filter.test.ts b/packages/spec/src/data/filter.test.ts index fd5ca8e8d0..5ae584f202 100644 --- a/packages/spec/src/data/filter.test.ts +++ b/packages/spec/src/data/filter.test.ts @@ -1093,9 +1093,20 @@ describe('parseFilterAST', () => { expect(parseFilterAST(['role', 'not_in', ['guest']])).toEqual({ role: { $nin: ['guest'] } }); }); - it('should convert contains/like operator', () => { + it('should convert contains operator', () => { expect(parseFilterAST(['name', 'contains', 'John'])).toEqual({ name: { $contains: 'John' } }); - expect(parseFilterAST(['name', 'like', 'John'])).toEqual({ name: { $contains: 'John' } }); + }); + + // [#7536] `like` used to be asserted here as a SECOND spelling of `contains`, + // which is the defect rather than the contract: `$contains` wraps its + // comparand in `%…%`, so folding `like` onto it made a caller's wildcards + // bind as literals and a wildcard-free pattern become a substring match. The + // pair now lowers to its own operators; the full repro table, the pattern + // language and the nested positions live in + // `filter-like-wire-lowering.test.ts`. + it('should convert like/ilike to their OWN operators, not to contains', () => { + expect(parseFilterAST(['name', 'like', 'John'])).toEqual({ name: { $like: 'John' } }); + expect(parseFilterAST(['name', 'ilike', 'john'])).toEqual({ name: { $ilike: 'john' } }); }); it('should convert notcontains/not_contains operator', () => { @@ -1288,7 +1299,7 @@ describe('isFilterAST', () => { describe('VALID_AST_OPERATORS', () => { it('should contain all standard comparison operators', () => { const expected = ['=', '==', '!=', '<>', '>', '>=', '<', '<=', 'in', 'nin', 'not_in', - 'contains', 'notcontains', 'not_contains', 'like', 'startswith', 'starts_with', 'endswith', 'ends_with', + 'contains', 'notcontains', 'not_contains', 'like', 'ilike', 'startswith', 'starts_with', 'endswith', 'ends_with', 'between', 'is_null', 'is_not_null']; for (const op of expected) { expect(VALID_AST_OPERATORS.has(op)).toBe(true); diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 81088eb512..618e68cd58 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -460,6 +460,39 @@ export const StringOperatorSchema = lazySchema(() => z.object({ + 'both transports); #6520 lowered it on every JS evaluation face, so it is ' + 'portable across every backend the platform ships.]' ), + + /** + * [#7536] Whole-string SQL `LIKE` pattern match — the comparand IS the + * pattern, and the CALLER binds the wildcards. The operator every `like` + * spelling of the AST vocabulary lowers to; unlike the `$contains` family + * above, nothing here is escaped or wrapped on the comparand's behalf. + */ + $like: z.string().optional().describe( + 'Whole-string pattern match with CALLER-bound wildcards: "%" matches any ' + + 'sequence (including empty), "_" matches exactly one character, and a ' + + 'backslash escapes the character after it ("\\%", "\\_", "\\\\") so it ' + + 'matches literally. The pattern must cover the WHOLE value — a pattern ' + + 'with no wildcards is an exact comparison, NOT a substring search; write ' + + '$contains for containment. A pattern ending in a lone unpaired backslash ' + + 'is refused (INVALID_FILTER). Comparison is case-SENSITIVE, same contract ' + + 'as $contains (#4706 Q2 = A); $ilike is the case-insensitive twin. ' + + '[#7536 declared it and lowered it on the SQL family; the JS evaluation ' + + 'faces refuse it loudly until their follow-up lands — see ' + + 'FILTER_OPERATORS for the staging.]' + ), + + /** + * [#7536] `$like`'s case-insensitive twin — same pattern language, folding + * ASCII case only (the #4706 Q1 = A domain `$icontains` pinned). + */ + $ilike: z.string().optional().describe( + 'Whole-string pattern match like $like — "%" / "_" wildcards bound by the ' + + 'caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) ' + + 'and ONLY ASCII case: "café" does NOT match "CAFÉ", the same #4706 Q1 = A ' + + 'boundary $icontains declares, because SQLite\'s fold is ASCII-only and ' + + 'three of the five backends are SQLite underneath. [#7536; staged with ' + + '$like — see FILTER_OPERATORS.]' + ), })); // ============================================================================ @@ -571,6 +604,163 @@ export function asciiCaseInsensitiveRegexSource(comparand: string): string { return out; } +// ============================================================================ +// The `$like` pattern language — ONE definition for every face (#7536) +// ============================================================================ + +/** + * [#7536] Does this `$like`/`$ilike` pattern end in a LONE, unpaired backslash? + * + * Such a pattern is REFUSED everywhere rather than given a meaning, because the + * backends could not be made to agree on one: Postgres raises `LIKE pattern + * must not end with escape character` at query time, while a JS translation + * would have to invent an answer ("literal backslash"? "dropped"?) that SQL + * then contradicts. A shape that cannot mean one thing on every backend is + * refused at the door — the same direction #5041/#5234 settled for + * uncompilable comparands. The check is shared so the refusal fires on the + * SAME patterns at every face; each face wraps it in its own ADR-0112 + * `INVALID_FILTER` envelope. + */ +export function hasDanglingLikeEscape(pattern: string): boolean { + let backslashes = 0; + for (let i = pattern.length - 1; i >= 0 && pattern[i] === '\\'; i--) backslashes++; + return backslashes % 2 === 1; +} + +/** + * [#7536] A `$like`/`$ilike` pattern as a regular-expression SOURCE with the + * SAME meaning — the one translation every JS evaluation face must share, for + * the reason {@link foldAsciiCase} gives: a translation written per package + * agrees with the spec on the day it is typed and never again. + * + * The pattern language, translated element by element: + * + * - `%` → `[\s\S]*` — any sequence, including empty and including newlines + * (SQL `LIKE` has no "dot-all" concept; `%` crosses line boundaries). + * - `_` → `[\s\S]` — exactly one character, any character. + * - `\x` → the character `x`, literally, whatever `x` is — this is how a + * caller matches a literal `%`, `_` or `\`. (Postgres and MySQL read an + * escaped ordinary character the same way.) + * - every other character → itself, regex-escaped, and — when `foldAscii` is + * set ($ilike) — an ASCII letter becomes its two-member character class + * (`[Aa]`), the {@link asciiCaseInsensitiveRegexSource} fold, so the fold + * lives in the pattern and callers pass NO regex flags (an `i` flag folds + * Unicode, which is the #4706 Q1 = A boundary violation). + * - the whole source is anchored `^…$`: `LIKE` matches the WHOLE value, so a + * wildcard-free pattern is an exact comparison, not a substring search. + * + * Throws a plain `Error` on a dangling trailing escape — gate with + * {@link hasDanglingLikeEscape} first to refuse in your own envelope; the + * throw here is the backstop that keeps a missed gate from minting a pattern + * with an invented meaning. + */ +export function likePatternToRegexSource(pattern: string, foldAscii = false): string { + if (hasDanglingLikeEscape(pattern)) { + throw new Error( + `LIKE pattern ${JSON.stringify(pattern)} ends with a lone unpaired backslash; ` + + 'write "\\\\\\\\" to match a literal backslash.', + ); + } + let out = '^'; + for (let i = 0; i < pattern.length; i++) { + let ch = pattern[i]; + let literal = false; + if (ch === '\\') { + // Guarded above: a trailing `\` cannot reach here. + ch = pattern[++i]; + literal = true; + } + if (!literal && ch === '%') { + out += '[\\s\\S]*'; + continue; + } + if (!literal && ch === '_') { + out += '[\\s\\S]'; + continue; + } + const code = ch.charCodeAt(0); + if (foldAscii && code >= ASCII_UPPER_FIRST && code <= ASCII_UPPER_LAST) { + out += `[${ch}${String.fromCharCode(code + ASCII_CASE_DELTA)}]`; + } else if ( + foldAscii + && code >= ASCII_UPPER_FIRST + ASCII_CASE_DELTA + && code <= ASCII_UPPER_LAST + ASCII_CASE_DELTA + ) { + out += `[${String.fromCharCode(code - ASCII_CASE_DELTA)}${ch}]`; + } else { + out += /[\\^$.*+?()[\]{}|/]/.test(ch) ? `\\${ch}` : ch; + } + } + return `${out}$`; +} + +/** + * [#7536] Does `value` match the `$like`/`$ilike` `pattern`? The predicate for + * every face that holds both strings in JS — the {@link likePatternToRegexSource} + * translation, evaluated. `foldAscii` selects the `$ilike` fold (ASCII only). + */ +export function matchesLikePattern(value: string, pattern: string, foldAscii = false): boolean { + return new RegExp(likePatternToRegexSource(pattern, foldAscii)).test(value); +} + +/** + * [#7536] A `$like`/`$ilike` pattern as an equivalent SQLite **GLOB** pattern. + * + * ## Why the SQLite family cannot just pass the pattern to `LIKE` + * + * `$like` is case-SENSITIVE (the #4706 Q2 = A contract its `$contains` sibling + * already answers), and SQLite's `LIKE` folds ASCII case unconditionally. + * #6518 measured every way out of that and landed on `GLOB`, which is + * case-exact by definition: `PRAGMA case_sensitive_like` is CONNECTION-global, + * so one query would redefine every other query on the connection, and + * `CAST(col AS BLOB) LIKE ?` was measured to match NOTHING. Three of the five + * backends are SQLite underneath (driver-sql on better-sqlite3, + * driver-sqlite-wasm, driver-turso on both transports), so without this + * translation `$like` would mean one thing on Postgres and another on SQLite — + * the divergence #6518 closed for `$contains`, re-opened one operator over. + * + * `GLOB` has a DIFFERENT pattern language, which is the whole reason this is a + * translation and not an escape: + * + * | LIKE | GLOB | note | + * |---|---|---| + * | `%` | `*` | any sequence, including empty | + * | `_` | `?` | exactly one character | + * | `\x` | `x`, escaped | the caller's literal, whatever `x` is | + * | `*` `?` `[` | `[*]` `[?]` `[[]` | GLOB's OWN metacharacters, which are ORDINARY characters to LIKE — this is the direction a hand-written escape forgets, and forgetting it is the `%`-matches-every-row bypass (#5567) wearing GLOB's clothes | + * | `]` | `]` | needs no escape: every `[` above becomes a class that closes itself, so no unclosed class survives for a later `]` to terminate | + * + * Shared from the spec rather than written per driver for the reason + * {@link foldAsciiCase} gives: `driver-sql`'s emitter and `driver-turso`'s + * remote transport compile the same operator independently, and a translation + * written twice agrees on the day it is typed and never again. + * + * Throws on a dangling trailing escape, exactly like + * {@link likePatternToRegexSource} — gate with {@link hasDanglingLikeEscape}. + */ +export function likePatternToGlobPattern(pattern: string): string { + if (hasDanglingLikeEscape(pattern)) { + throw new Error( + `LIKE pattern ${JSON.stringify(pattern)} ends with a lone unpaired backslash; ` + + 'write "\\\\\\\\" to match a literal backslash.', + ); + } + const globEscape = (ch: string) => (/[*?[]/.test(ch) ? `[${ch}]` : ch); + let out = ''; + for (let i = 0; i < pattern.length; i++) { + const ch = pattern[i]; + if (ch === '\\') { + // Guarded above: a trailing `\` cannot reach here. + out += globEscape(pattern[++i]); + continue; + } + if (ch === '%') { out += '*'; continue; } + if (ch === '_') { out += '?'; continue; } + out += globEscape(ch); + } + return out; +} + // ============================================================================ // 3.5 Special Operators // ============================================================================ @@ -634,6 +824,11 @@ export const FieldOperatorsSchema = lazySchema(() => z.object({ $startsWith: z.string().optional(), $endsWith: z.string().optional(), $icontains: z.string().optional(), + // Pattern-matching pair (#7536): the comparand IS a LIKE pattern, wildcards + // bound by the CALLER — see {@link StringOperatorSchema} for the language. + // `$ilike` folds ASCII case only, `$like` is case-exact. + $like: z.string().optional(), + $ilike: z.string().optional(), // Special $null: z.boolean().optional(), @@ -1009,7 +1204,17 @@ const AST_OPERATOR_MAP = { 'contains': '$contains', 'notcontains': '$notContains', 'not_contains': '$notContains', - 'like': '$contains', + // [#7536] `like`/`ilike` lower to their OWN operators, not `$contains`. + // The former `'like': '$contains'` entry silently rewrote what the query + // means: `$contains` LIKE-escapes the comparand and wraps it in `%…%`, so a + // caller's own wildcards bound as literals (`%Industries` matched nothing) + // and a wildcard-free comparand became a substring match nobody asked for. + // `canonicalAstOperator` below always exempted these two spellings for + // exactly that reason; the lowering the wire path takes now agrees with it. + // (`ilike` had NO entry at all, so `isFilterAST` refused it — it enters the + // vocabulary here with its lowering, per the #3948 single-table rule.) + 'like': '$like', + 'ilike': '$ilike', 'startswith': '$startsWith', 'starts_with': '$startsWith', 'endswith': '$endsWith', @@ -1066,6 +1271,7 @@ const CANONICAL_INFIX: Record = { '$in': 'in', '$nin': 'nin', '$contains': 'contains', '$notContains': 'not_contains', '$startsWith': 'starts_with', '$endsWith': 'ends_with', '$between': 'between', + '$like': 'like', '$ilike': 'ilike', }; export function canonicalAstOperator(op: string): string { @@ -1081,11 +1287,12 @@ export function canonicalAstOperator(op: string): string { ) { return 'is_not_null'; } - // `like`/`ilike` share the `$contains` lowering but are NOT substring matches - // at the driver: driver-sql passes them to SQL verbatim, so the caller binds - // the wildcards. Folding them onto `contains` would silently wrap the value in - // `%…%` and change what the query means. - if (lower === 'like' || lower === 'ilike') return lower; + // `like`/`ilike` used to need a hand-written exemption here: they SHARED the + // `$contains` lowering while not being substring matches, so the generic + // round-trip below would have folded them onto `contains` and silently + // wrapped the value in `%…%`. #7536 gave them their own lowerings + // (`$like`/`$ilike`), so the generic path now answers `like`/`ilike` by + // construction — the exemption is retired, not the rule it protected. const dollar = astOperatorLowering(lower); if (!dollar) return lower; return CANONICAL_INFIX[dollar] ?? lower; @@ -1465,6 +1672,31 @@ export const FilterArraySchema: z.ZodType = z.lazy(() * declaration and enforcement surfaces, so a name added to one and not the other * fails in either direction. * + * ## `$like` / `$ilike` are STAGED here, on purpose (#7536) + * + * They are declared by {@link StringOperatorSchema} and + * {@link FieldOperatorsSchema} and deliberately ABSENT from this array, which + * is the staging direction the `$icontains` paragraph above prescribes — and + * for exactly the mechanism it measured. Membership here is what makes + * `driver-memory`'s `SUPPORTED_FIELD_OPERATORS` ACCEPT a name; adding `$like` + * before that driver's matcher has an arm would turn its loud refusal into a + * dropped predicate, i.e. every row. + * + * What is implemented today, and what refuses: + * + * | face | `$like` / `$ilike` | + * |---|---| + * | `driver-sql` (and `driver-sqlite-wasm`, which inherits its compiler) | ANSWERS — `LIKE` / `GLOB` per dialect, caller-bound wildcards | + * | `driver-turso` — local (inherits `SqlDriver`) and remote (its own compiler) | ANSWERS on both transports | + * | `@objectstack/formula` `matchesFilterCondition` | ANSWERS — {@link matchesLikePattern}, so a write-side `check` agrees with the read-side SQL | + * | `driver-memory`, `driver-mongodb`, `objectql` `having`, `service-analytics` | REFUSE, loudly, in the ADR-0112 `INVALID_FILTER` envelope — because THIS array does not name the operator | + * + * That split is the point rather than a gap: #7536 exists because a `like` + * predicate was being SILENTLY given `$contains`' meaning, and a face that + * quietly answers a different question is strictly worse than one that + * refuses. Clearing the staging means arms on the remaining faces in ONE PR, + * the #6520 direction — tracked as the follow-up filed on #7536. + * * Retired operators (`$regex`, `$options`) are not here either, and never were. * Their prescriptions live in {@link RETIRED_FILTER_OPERATORS}. */ From f1bee9d1bacbe3d8eb87e3fb52ff9981e11c9b26 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 11 Aug 2026 06:25:38 +0000 Subject: [PATCH 2/4] test: retarget $like unknown-operator exemplars, docs (#7536) Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01PhHptz16p1kRmbmuzEZkgd --- .../src/remote-transport-not-operator.test.ts | 11 +++++++++-- packages/spec/src/contracts/data-driver.test.ts | 15 +++++++++++---- packages/spec/src/contracts/data-driver.ts | 3 ++- 3 files changed, 22 insertions(+), 7 deletions(-) diff --git a/packages/drivers/driver-turso/src/remote-transport-not-operator.test.ts b/packages/drivers/driver-turso/src/remote-transport-not-operator.test.ts index f727680863..a8b39dc9e4 100644 --- a/packages/drivers/driver-turso/src/remote-transport-not-operator.test.ts +++ b/packages/drivers/driver-turso/src/remote-transport-not-operator.test.ts @@ -368,9 +368,16 @@ describe('RemoteTransport $not (#1076)', () => { /compiles to NO predicate/, ); // #1004: an unknown operator. + // + // [#7536] The exemplar used to be `$like`, and that stopped being a valid + // choice: `$like` is now a DECLARED operator this transport compiles, so + // the assertion was pinning a refusal that must no longer happen. + // `$sounds_like` is this repo's standing stand-in for a spelling the + // protocol does not have and will not grow — the same name `sql-driver.ts` + // reaches for when it needs one. await expect( - t.find('deal', { where: { $not: { stage: { $like: 'w%' } } } } as unknown as QueryAST), - ).rejects.toThrow(/Unsupported filter operator "\$like"/); + t.find('deal', { where: { $not: { stage: { $sounds_like: 'w%' } } } } as unknown as QueryAST), + ).rejects.toThrow(/Unsupported filter operator "\$sounds_like"/); // #1058: an unbindable comparand. await expect( t.find('deal', { where: { $not: { amount: { $gt: { $field: 'budget' } } } } } as unknown as QueryAST), diff --git a/packages/spec/src/contracts/data-driver.test.ts b/packages/spec/src/contracts/data-driver.test.ts index 89b0bbebb3..0f74855c67 100644 --- a/packages/spec/src/contracts/data-driver.test.ts +++ b/packages/spec/src/contracts/data-driver.test.ts @@ -256,10 +256,17 @@ describe('IDataDriver', () => { // fix than it delivers: `where` is `FilterCondition`, whose index // signature is `[key: string]: any` because ANY field name is a legal key. // An operator the dialect does not have is therefore still not a type - // error — `$like` (cloud#1030) reaches the runtime filter compiler and is - // rejected there, not here. Removing the cast does not close that door; - // only a closed operator vocabulary would, which is a separate change. - const unknownOperator: DriverQuery = { where: { name: { $like: 'acme%' } } }; + // error — it reaches the runtime filter compiler and is rejected there, + // not here. Removing the cast does not close that door; only a closed + // operator vocabulary would, which is a separate change. + // + // [#7536] The exemplar was `$like` (the operator cloud#1030 measured + // reaching the runtime). It is a DECLARED operator now, so it no longer + // illustrates "an operator the dialect does not have" — the point stands, + // the example had to move to a spelling that is still undeclared. The + // history is untouched: `$like` is what cloud#1030 caught, back when it + // was not in the protocol. + const unknownOperator: DriverQuery = { where: { name: { $sounds_like: 'acme%' } } }; expect(unknownOperator.where).toBeTruthy(); }); }); diff --git a/packages/spec/src/contracts/data-driver.ts b/packages/spec/src/contracts/data-driver.ts index e30d88155a..0ea8233bda 100644 --- a/packages/spec/src/contracts/data-driver.ts +++ b/packages/spec/src/contracts/data-driver.ts @@ -18,7 +18,8 @@ import type { QueryAST } from '../data/query.zod.js'; * was paid for in blanket casts instead: a direct caller holding only a `where` * could not name a type for it, reached for `as any`, and lost `where`'s type * checking along with the object name — which is how an operator the filter - * dialect does not have (`$like`) survived compilation and reached the runtime + * dialect did not have (`$like`, undeclared then; #7536 has since made it a + * real one) survived compilation and reached the runtime * (objectstack#5181, cloud#1053, cloud#1030). * * What this deliberately does NOT drop is the `object` inside an `expand` From b157e8cf46ef84c67f6e2d592190791c71695c07 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 11 Aug 2026 06:37:47 +0000 Subject: [PATCH 3/4] chore(spec): regenerate api-surface, export-origins and reference docs (#7536) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The four new pattern-language exports and the two new declared operators. Also corrects the $like describe() and the FILTER_OPERATORS staging table, which both said the JS faces refuse — driver-memory and formula answer. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01PhHptz16p1kRmbmuzEZkgd --- .changeset/like-wire-lowering.md | 5 ++++- content/docs/references/data/filter.mdx | 2 ++ packages/spec/api-surface/data.json | 4 ++++ packages/spec/export-origins/data.json | 4 ++++ packages/spec/src/data/filter.zod.ts | 12 ++++++++---- 5 files changed, 22 insertions(+), 5 deletions(-) diff --git a/.changeset/like-wire-lowering.md b/.changeset/like-wire-lowering.md index d4ddd91693..2ed1c6e8d7 100644 --- a/.changeset/like-wire-lowering.md +++ b/.changeset/like-wire-lowering.md @@ -96,7 +96,10 @@ every face, by one shared test. `like` is now an exact match. 2. **`like`/`ilike` on `driver-mongodb`, objectql `having` and analytics now return `400 INVALID_FILTER`** where a (wrong) substring answer came back - before. Write `$contains`/`$icontains` on those backends. + before. Write `$contains`/`$icontains` on those backends. `driver-memory` is + deliberately NOT in that list — it implements the operators, because an + application whose tests run on the in-memory double and whose production runs + SQL must not meet a 400 in test for a filter that works in production. 3. **`@objectstack/client`'s `.contains()`, `.startsWith()` and `.endsWith()` emit different operators.** They used to build a `like` tuple by gluing wildcards onto the caller's value (`[field, 'like', '%' + value + '%']`), diff --git a/content/docs/references/data/filter.mdx b/content/docs/references/data/filter.mdx index 991f370cef..f63978951d 100644 --- a/content/docs/references/data/filter.mdx +++ b/content/docs/references/data/filter.mdx @@ -148,6 +148,8 @@ Type: `[FilterArray](#filterarray)[]` | **$startsWith** | `string` | optional | | | **$endsWith** | `string` | optional | | | **$icontains** | `string` | optional | Contains substring, ignoring case — but ONLY ASCII case (A-Z against a-z). Every other character compares literally, so "café" does NOT match "CAFÉ" and "москва" does not match "МОСКВА". The domain is ASCII because that is the one fold all five backends can deliver: SQLite (and therefore turso and sqlite-wasm) folds ASCII only, so a Unicode promise here would be a guarantee three of the five could not keep. The comparand is matched LITERALLY — "%", "_" and regex metacharacters are ordinary characters, not wildcards. Case-SENSITIVE containment is $contains. [#5701 declared it; #5702 lowered it on the SQL family (driver-sql, driver-sqlite-wasm, driver-turso on both transports); #6520 lowered it on every JS evaluation face, so it is portable across every backend the platform ships.] | +| **$like** | `string` | optional | Whole-string pattern match with CALLER-bound wildcards: "%" matches any sequence (including empty), "_" matches exactly one character, and a backslash escapes the character after it ("\\%", "\\_", "\\\\") so it matches literally. The pattern must cover the WHOLE value — a pattern with no wildcards is an exact comparison, NOT a substring search; write $contains for containment. A pattern ending in a lone unpaired backslash is refused (INVALID_FILTER). Comparison is case-SENSITIVE, same contract as $contains (#4706 Q2 = A); $ilike is the case-insensitive twin. [#7536. Answered by the SQL family (driver-sql, driver-sqlite-wasm, driver-turso on both transports), by driver-memory and by @objectstack/formula. driver-mongodb, objectql `having` and service-analytics REFUSE it in the INVALID_FILTER envelope rather than approximating it — see FILTER_OPERATORS for why it is staged out of that allowlist.] | +| **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same #4706 Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. [#7536; staged with $like — see FILTER_OPERATORS.] | --- diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index c2a6e2cbb3..7c34172f05 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -629,6 +629,7 @@ "getSqliteConfigJsonSchema (const)", "getSqliteWasmConfigJsonSchema (const)", "getTursoConfigJsonSchema (const)", + "hasDanglingLikeEscape (function)", "hasDynamicTokens (function)", "hookForm (const)", "isApiOperationAllowed (function)", @@ -654,7 +655,10 @@ "isTitleEligible (function)", "isUniqueDeclared (function)", "isVirtualSearchField (function)", + "likePatternToGlobPattern (function)", + "likePatternToRegexSource (function)", "lintAuthoredRecordKeys (function)", + "matchesLikePattern (function)", "missingFieldValues (function)", "nextUtcCalendarDay (function)", "objectForm (const)", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index 72ff514e1c..601ca826fa 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -629,6 +629,7 @@ "getSqliteConfigJsonSchema": "src/data/driver/sqlite.zod.ts#getSqliteConfigJsonSchema (const)", "getSqliteWasmConfigJsonSchema": "src/data/driver/sqlite.zod.ts#getSqliteWasmConfigJsonSchema (const)", "getTursoConfigJsonSchema": "src/data/driver/turso.zod.ts#getTursoConfigJsonSchema (const)", + "hasDanglingLikeEscape": "src/data/filter.zod.ts#hasDanglingLikeEscape (function)", "hasDynamicTokens": "src/data/autonumber-format.ts#hasDynamicTokens (function)", "hookForm": "src/data/hook.form.ts#hookForm (const)", "isApiOperationAllowed": "src/data/api-derivation.ts#isApiOperationAllowed (function)", @@ -654,7 +655,10 @@ "isTitleEligible": "src/data/display-name.ts#isTitleEligible (function)", "isUniqueDeclared": "src/data/field.zod.ts#isUniqueDeclared (function)", "isVirtualSearchField": "src/data/search-fields.ts#isVirtualSearchField (function)", + "likePatternToGlobPattern": "src/data/filter.zod.ts#likePatternToGlobPattern (function)", + "likePatternToRegexSource": "src/data/filter.zod.ts#likePatternToRegexSource (function)", "lintAuthoredRecordKeys": "src/data/authoring-key-lint.ts#lintAuthoredRecordKeys (function)", + "matchesLikePattern": "src/data/filter.zod.ts#matchesLikePattern (function)", "missingFieldValues": "src/data/autonumber-format.ts#missingFieldValues (function)", "nextUtcCalendarDay": "src/data/calendar-day.ts#nextUtcCalendarDay (function)", "objectForm": "src/data/object.form.ts#objectForm (const)", diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index 618e68cd58..1be2baa6c2 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -476,9 +476,12 @@ export const StringOperatorSchema = lazySchema(() => z.object({ + '$contains for containment. A pattern ending in a lone unpaired backslash ' + 'is refused (INVALID_FILTER). Comparison is case-SENSITIVE, same contract ' + 'as $contains (#4706 Q2 = A); $ilike is the case-insensitive twin. ' - + '[#7536 declared it and lowered it on the SQL family; the JS evaluation ' - + 'faces refuse it loudly until their follow-up lands — see ' - + 'FILTER_OPERATORS for the staging.]' + + '[#7536. Answered by the SQL family (driver-sql, driver-sqlite-wasm, ' + + 'driver-turso on both transports), by driver-memory and by ' + + '@objectstack/formula. driver-mongodb, objectql `having` and ' + + 'service-analytics REFUSE it in the INVALID_FILTER envelope rather than ' + + 'approximating it — see FILTER_OPERATORS for why it is staged out of that ' + + 'allowlist.]' ), /** @@ -1688,8 +1691,9 @@ export const FilterArraySchema: z.ZodType = z.lazy(() * |---|---| * | `driver-sql` (and `driver-sqlite-wasm`, which inherits its compiler) | ANSWERS — `LIKE` / `GLOB` per dialect, caller-bound wildcards | * | `driver-turso` — local (inherits `SqlDriver`) and remote (its own compiler) | ANSWERS on both transports | + * | `driver-memory` — query path and reference matcher | ANSWERS — it widens its own `SUPPORTED_FIELD_OPERATORS` by hand, the way `driver-turso`'s remote transport has carried `$icontains` since #5702. It is the in-memory DOUBLE: an app whose tests run there and whose production runs SQL must not get a 400 for a filter that works | * | `@objectstack/formula` `matchesFilterCondition` | ANSWERS — {@link matchesLikePattern}, so a write-side `check` agrees with the read-side SQL | - * | `driver-memory`, `driver-mongodb`, `objectql` `having`, `service-analytics` | REFUSE, loudly, in the ADR-0112 `INVALID_FILTER` envelope — because THIS array does not name the operator | + * | `driver-mongodb`, `objectql` `having`, `service-analytics` | REFUSE, loudly, in the ADR-0112 `INVALID_FILTER` envelope — they derive acceptance from THIS array, which does not name the operator | * * That split is the point rather than a gap: #7536 exists because a `like` * predicate was being SILENTLY given `$contains`' meaning, and a face that From f05d9f3ea49cc2b385ee56481dc441a553842c3b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 11 Aug 2026 06:43:51 +0000 Subject: [PATCH 4/4] docs(objectql): document $like/$ilike in the query-syntax operator table (#7536) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the two operators to the reference table and a short section on why $like is not a spelling of $contains — text vs pattern, substring vs whole value — plus the per-backend coverage split. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01PhHptz16p1kRmbmuzEZkgd --- .../docs/protocol/objectql/query-syntax.mdx | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/content/docs/protocol/objectql/query-syntax.mdx b/content/docs/protocol/objectql/query-syntax.mdx index e1a112dff7..e92cd92606 100644 --- a/content/docs/protocol/objectql/query-syntax.mdx +++ b/content/docs/protocol/objectql/query-syntax.mdx @@ -254,10 +254,44 @@ const query: QueryAST = { | `$notContains` | String does not contain, **case-sensitive** | `{ name: { $notContains: 'test' } }` | | `$startsWith` | String starts with, **case-sensitive** | `{ email: { $startsWith: 'admin' } }` | | `$endsWith` | String ends with, **case-sensitive** | `{ domain: { $endsWith: '.com' } }` | +| `$like` | `LIKE` pattern — **you write the wildcards**, **case-sensitive** | `{ name: { $like: '%Industries' } }` | +| `$ilike` | Same pattern language, **ignoring ASCII case** | `{ name: { $ilike: '%industries' } }` | | `$between` | Range (inclusive) | `{ close_date: { $between: ['2024-01-01', '2024-12-31'] } }` | | `$null` | Null check | `{ manager_id: { $null: true } }` / `{ phone: { $null: false } }` | | `$exists` | Field exists (NoSQL) | `{ metadata: { $exists: true } }` | +### `$like` is not a spelling of `$contains` + +The two take different things and picking the wrong one is a silently wrong +answer rather than an error: + +- **`$contains` takes TEXT.** It is matched literally as a substring, and `%`, + `_` and regex metacharacters in your comparand are ordinary characters — + escaped on your behalf. +- **`$like` takes a PATTERN.** `%` matches any sequence, `_` matches exactly one + character, and a backslash escapes either. The pattern is matched against the + **whole value**, so a pattern with no wildcards is an *exact comparison*, not + a substring search. + +```ts +{ name: { $contains: 'Industries' } } // Acme Industries, Industries Ltd, Industries +{ name: { $like: 'Industries' } } // Industries — exact +{ name: { $like: '%Industries' } } // Acme Industries, Industries — ends with +``` + +A pattern ending in a lone unpaired backslash is refused (`INVALID_FILTER`): no +reading of it survives every backend, so it is rejected rather than guessed. + + + **Backend coverage.** `$like` / `$ilike` are executed by the SQL family + (`driver-sql`, `driver-sqlite-wasm`, `driver-turso` on both transports), + `driver-memory`, and the in-memory `matchesFilter` evaluator. + `driver-mongodb`, ObjectQL `having` and the analytics compilers **refuse** + them with `INVALID_FILTER` rather than approximating them — use `$contains` / + `$icontains` there. That split is deliberate: a backend that quietly answered + a *different* question is the defect these operators exist to end (#7536). + + ### Case Sensitivity The string operators compare **case-sensitively**. `$icontains` is the one that does