Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions .changeset/like-wire-lowering.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
---
"@objectstack/spec": minor
"@objectstack/driver-sql": minor
"@objectstack/driver-turso": minor
"@objectstack/driver-memory": minor
"@objectstack/formula": minor
"@objectstack/client": patch
---

fix(spec,drivers,formula,client): `like`/`ilike` stop being folded onto `$contains` at the wire (#7536)

A `like` predicate that arrived over HTTP was rewritten into a substring search
before any driver saw it, because `AST_OPERATOR_MAP` (`data/filter.zod.ts`)
carried `'like': '$contains'`. `$contains` LIKE-escapes its comparand and wraps
it in `%…%`, which breaks a `like` in **both** directions at once. Measured in
QA run #7463 against showcase on SQLite:

| filter | before | now |
|---|---|---|
| `["name","like","%Industries"]` | `200`, **0 rows** — the `%` bound as a literal percent sign | the rows ENDING WITH `Industries` |
| `["name","like","Industries"]` | a substring match, **byte-identical to the `$contains` control** | an EXACT match |
| `["name","ilike","…"]` | `400` — `ilike` had no lowering at all, so `isFilterAST()` refused the whole filter | the case-insensitive twin |

The second row is the tell: `like` and `$contains` producing the same bytes
means `like` was not reaching the driver as a pattern at all.

The file already documented the contract being violated. `canonicalAstOperator`,
thirty lines below the map entry, carried a hand-written exemption for
`like`/`ilike` whose comment read: *"they are NOT substring matches at the
driver: driver-sql passes them to SQL verbatim, so the caller binds the
wildcards. Folding them onto `contains` would silently wrap the value in `%…%`
and change what the query means."* That exemption only ever shaped its own
output; the lowering the wire path takes had none. A consequence worth naming:
driver-sql's `like`/`ilike` handling has been unreachable from the wire since
#5158.

## What changed

**New operators `$like` / `$ilike`** on `StringOperatorSchema` and
`FieldOperatorsSchema`. The comparand IS the pattern: `%` matches any sequence,
`_` matches exactly one character, a backslash escapes either, and the pattern
must cover the WHOLE value — so a pattern with no wildcards is an exact
comparison, not a substring search. `$like` is case-SENSITIVE (the #4706 Q2 = A
contract its `$contains` sibling answers); `$ilike` folds ASCII case and nothing
else (Q1 = A), so `café` does not match `CAFÉ`.

`AST_OPERATOR_MAP` now lowers `like` → `$like` and `ilike` → `$ilike`. `ilike`
enters the AST vocabulary for the first time — it previously had no entry, so
`isFilterAST()` refused it. `canonicalAstOperator`'s hand-written exemption is
retired: the generic round-trip answers `like`/`ilike` by construction now, so
the special case is gone along with the reason it existed.

The pattern language is defined **once**, in the spec, and shared by every face
that needs it — `hasDanglingLikeEscape`, `likePatternToRegexSource`,
`matchesLikePattern` and `likePatternToGlobPattern`. Six faces implementing one
pattern language separately is the `#3948` shape reached through translation
instead of vocabulary.

**Which backends answer, and which refuse.** `$like`/`$ilike` are deliberately
NOT in `FILTER_OPERATORS`, the runtime allowlist several packages derive
acceptance from — adding a name there before every face has an arm turns a loud
refusal into a silently DROPPED predicate, which is the widening measured in
#5701 and ruled on in #3948.

| face | `$like` / `$ilike` |
|---|---|
| `driver-sql` (and `driver-sqlite-wasm`, which inherits its compiler) | **answers** — `LIKE` on Postgres/MySQL, `GLOB` on SQLite |
| `driver-turso`, both transports | **answers** — the remote transport compiles independently, holds to the local one by a parity suite |
| `driver-memory`, both faces | **answers** — the in-memory double must not 400 for a filter that works in production |
| `@objectstack/formula` (`matchesFilterCondition`) | **answers** — so a write-side RLS `check` agrees with the read-side SQL |
| `driver-mongodb`, objectql `having`, `service-analytics` | **refuse**, loudly, in the ADR-0112 `INVALID_FILTER` envelope |

The refusals are the point rather than a gap: #7536 exists because a `like` was
silently given `$contains`' meaning, and a face that quietly answers a different
question is worse than one that refuses. Clearing the remainder means arms on
those faces in one PR — the #6520 direction.

**Why SQLite gets `GLOB`.** `$like` is case-exact and SQLite's `LIKE` folds
ASCII unconditionally, which cannot be switched off per statement
(`PRAGMA case_sensitive_like` is connection-global). That is #6518's finding,
and the operator it landed on. Because GLOB speaks a different pattern language
(`*`/`?`, and `%`/`_` are ordinary characters), the pattern is TRANSLATED rather
than escaped — including GLOB's own metacharacters, which are ordinary to LIKE:
an unescaped `*` in a GLOB pattern is the same filter bypass an unescaped `%` is
under LIKE (#5567).

**Refused rather than given a meaning:** a pattern ending in a lone unpaired
backslash. No reading survives every backend — Postgres rejects such a pattern
outright, GLOB has no escape character at all — so it is refused at the door on
every face, by one shared test.

## ⚠️ Behaviour changes

1. **`like` now means `LIKE`.** If you were relying on `like` behaving as a
substring search — the defect — write `contains` instead. A wildcard-free
`like` is now an exact match.
2. **`like`/`ilike` on `driver-mongodb`, objectql `having` and analytics now
return `400 INVALID_FILTER`** where a (wrong) substring answer came back
before. Write `$contains`/`$icontains` on those backends. `driver-memory` is
deliberately NOT in that list — it implements the operators, because an
application whose tests run on the in-memory double and whose production runs
SQL must not meet a 400 in test for a filter that works in production.
3. **`@objectstack/client`'s `.contains()`, `.startsWith()` and `.endsWith()`
emit different operators.** They used to build a `like` tuple by gluing
wildcards onto the caller's value (`[field, 'like', '%' + value + '%']`),
which was wrong twice over: the wire folded `like` onto `$contains`, which
escaped the glued `%` back into a literal, so `.contains('name','Corp')`
searched for the text `%Corp%` and matched only rows containing percent
signs. And once `like` reaches the driver as a real pattern, the glue becomes
the *other* bug — a `%` or `_` inside the caller's own value would silently
become a wildcard. They now emit `contains` / `starts_with` / `ends_with`,
whose comparand is text. `.like()` is unchanged and finally works; `.ilike()`
is new.

Note the case semantics this corrects on paper too: `.contains()`'s docblock
claimed "case-insensitive", but the `$contains` family is case-SENSITIVE by
contract (#4706 Q2 = A). Use `.ilike()` for a case-insensitive pattern.
14 changes: 11 additions & 3 deletions content/docs/api/client-sdk.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -473,14 +473,22 @@ const filter = createFilter()
.greaterThan('revenue', 10000) // field > value
.lessThanOrEqual('age', 100) // field <= value
.in('category', ['A', 'B', 'C']) // field IN (...)
.like('name', '%Corp%') // field LIKE pattern
.contains('name', 'Corp') // LIKE %Corp%
.startsWith('name', 'Acme') // LIKE Acme%
.like('name', '%Corp%') // pattern match — the wildcards are YOURS
.ilike('name', '%corp%') // same, ignoring ASCII case
.contains('name', 'Corp') // literal substring — no wildcards involved
.startsWith('name', 'Acme') // literal prefix
.isNull('deleted_at') // field IS NULL
.between('created_at', '2024-01', '2024-12')
.build();
```

`like` / `ilike` take a **pattern**: `%` matches any sequence, `_` matches
exactly one character, and a backslash escapes either. The pattern is matched
against the whole value, so `like('name', 'Corp')` is an exact comparison — for
a substring search use `contains`, whose argument is plain text and needs no
escaping. `like` / `ilike` are executed by the SQL backends; the others refuse
them with `invalid_filter`.

---

## Query Options
Expand Down
22 changes: 20 additions & 2 deletions content/docs/deployment/troubleshooting.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -125,7 +125,8 @@ The custom error map provides "Did you mean?" suggestions for common typos.
```
$eq, $ne, $gt, $gte, $lt, $lte,
$in, $nin, $between,
$contains, $notContains, $startsWith, $endsWith,
$contains, $notContains, $startsWith, $endsWith, $icontains,
$like, $ilike,
$null, $exists,
$and, $or, $not
```
Expand All@@ -135,10 +136,27 @@ Common mistakes:
|:---|:---|
| `$equal` | `$eq` |
| `$notEqual` | `$ne` |
| `$like` | `$contains` |
| `$greaterThan` | `$gt` |
| `$isNull` | `$null` |
| `$notIn` | `$nin` |
| `$regex` | `$icontains` |

`$like` is **not** a misspelling of `$contains` — the two mean different things,
and picking the wrong one gives a silently wrong answer rather than an error:

- `$contains` takes **text**, matched literally as a substring. A `%` or `_` in
the comparand is an ordinary character.
- `$like` takes a **pattern**, matched against the whole value, with the
wildcards you write: `%` is any sequence, `_` is exactly one character, and a
backslash escapes either. A pattern with no wildcards is an exact comparison,
not a substring search.

`$ilike` is `$like`'s case-insensitive twin, and `$icontains` is `$contains`'s.
Both fold ASCII case only, so `café` does not match `CAFÉ`.

`$like` / `$ilike` are executed by the SQL family (`driver-sql`,
`driver-sqlite-wasm`, `driver-turso`). The other backends refuse them with this
same `invalid_filter` error — write `$contains` there.

---

Expand Down
34 changes: 34 additions & 0 deletions content/docs/protocol/objectql/query-syntax.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -254,10 +254,44 @@ const query: QueryAST = {
| `$notContains` | String does not contain, **case-sensitive** | `{ name: { $notContains: 'test' } }` |
| `$startsWith` | String starts with, **case-sensitive** | `{ email: { $startsWith: 'admin' } }` |
| `$endsWith` | String ends with, **case-sensitive** | `{ domain: { $endsWith: '.com' } }` |
| `$like` | `LIKE` pattern — **you write the wildcards**, **case-sensitive** | `{ name: { $like: '%Industries' } }` |
| `$ilike` | Same pattern language, **ignoring ASCII case** | `{ name: { $ilike: '%industries' } }` |
| `$between` | Range (inclusive) | `{ close_date: { $between: ['2024-01-01', '2024-12-31'] } }` |
| `$null` | Null check | `{ manager_id: { $null: true } }` / `{ phone: { $null: false } }` |
| `$exists` | Field exists (NoSQL) | `{ metadata: { $exists: true } }` |

### `$like` is not a spelling of `$contains`

The two take different things and picking the wrong one is a silently wrong
answer rather than an error:

- **`$contains` takes TEXT.** It is matched literally as a substring, and `%`,
`_` and regex metacharacters in your comparand are ordinary characters —
escaped on your behalf.
- **`$like` takes a PATTERN.** `%` matches any sequence, `_` matches exactly one
character, and a backslash escapes either. The pattern is matched against the
**whole value**, so a pattern with no wildcards is an *exact comparison*, not
a substring search.

```ts
{ name: { $contains: 'Industries' } } // Acme Industries, Industries Ltd, Industries
{ name: { $like: 'Industries' } } // Industries — exact
{ name: { $like: '%Industries' } } // Acme Industries, Industries — ends with
```

A pattern ending in a lone unpaired backslash is refused (`INVALID_FILTER`): no
reading of it survives every backend, so it is rejected rather than guessed.

<Callout type="warn">
**Backend coverage.** `$like` / `$ilike` are executed by the SQL family
(`driver-sql`, `driver-sqlite-wasm`, `driver-turso` on both transports),
`driver-memory`, and the in-memory `matchesFilter` evaluator.
`driver-mongodb`, ObjectQL `having` and the analytics compilers **refuse**
them with `INVALID_FILTER` rather than approximating them — use `$contains` /
`$icontains` there. That split is deliberate: a backend that quietly answered
a *different* question is the defect these operators exist to end (#7536).
</Callout>

### Case Sensitivity

The string operators compare **case-sensitively**. `$icontains` is the one that does
Expand Down
2 changes: 2 additions & 0 deletions content/docs/references/data/filter.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -148,6 +148,8 @@ Type: `[FilterArray](#filterarray)[]`
| **$startsWith** | `string` | optional | |
| **$endsWith** | `string` | optional | |
| **$icontains** | `string` | optional | Contains substring, ignoring case — but ONLY ASCII case (A-Z against a-z). Every other character compares literally, so "café" does NOT match "CAFÉ" and "москва" does not match "МОСКВА". The domain is ASCII because that is the one fold all five backends can deliver: SQLite (and therefore turso and sqlite-wasm) folds ASCII only, so a Unicode promise here would be a guarantee three of the five could not keep. The comparand is matched LITERALLY — "%", "_" and regex metacharacters are ordinary characters, not wildcards. Case-SENSITIVE containment is $contains. [#5701 declared it; #5702 lowered it on the SQL family (driver-sql, driver-sqlite-wasm, driver-turso on both transports); #6520 lowered it on every JS evaluation face, so it is portable across every backend the platform ships.] |
| **$like** | `string` | optional | Whole-string pattern match with CALLER-bound wildcards: "%" matches any sequence (including empty), "_" matches exactly one character, and a backslash escapes the character after it ("\\%", "\\_", "\\\\") so it matches literally. The pattern must cover the WHOLE value — a pattern with no wildcards is an exact comparison, NOT a substring search; write $contains for containment. A pattern ending in a lone unpaired backslash is refused (INVALID_FILTER). Comparison is case-SENSITIVE, same contract as $contains (#4706 Q2 = A); $ilike is the case-insensitive twin. [#7536. Answered by the SQL family (driver-sql, driver-sqlite-wasm, driver-turso on both transports), by driver-memory and by @objectstack/formula. driver-mongodb, objectql `having` and service-analytics REFUSE it in the INVALID_FILTER envelope rather than approximating it — see FILTER_OPERATORS for why it is staged out of that allowlist.] |
| **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same #4706 Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. [#7536; staged with $like — see FILTER_OPERATORS.] |


---
Expand Down
41 changes: 37 additions & 4 deletions packages/client/src/client.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -1015,25 +1015,58 @@ describe('FilterBuilder enhancements', () => {
expect(f[2]).toEqual(['age', '<=', 65]);
});

it('should add contains filter', () => {
// [#7536] These three used to build a `like` tuple by gluing wildcards onto
// the caller's value (`['name', 'like', '%alice%']`), which was wrong in two
// directions at once. While the wire folded `like` onto `$contains` — which
// LIKE-ESCAPES its comparand — the glued `%` came back as a literal percent
// sign, so `.contains('name','alice')` searched for the text `%alice%` and
// matched only rows containing percent signs. And once `like` reaches the
// driver as a real pattern, the glue becomes the OTHER bug: a `%` or `_`
// inside the caller's own value would silently become a wildcard. Naming the
// operator that means what the method says removes both.
it('should add contains filter as the literal-text operator', () => {
const f = createFilter<{ name: string }>()
.contains('name', 'alice')
.build();
expect(f).toEqual(['name', 'like', '%alice%']);
expect(f).toEqual(['name', 'contains', 'alice']);
});

it('should not let the caller\'s own wildcards leak into a contains filter', () => {
// The regression this shape prevents: `50%` is TEXT here, not a pattern.
const f = createFilter<{ name: string }>()
.contains('name', '50%')
.build();
expect(f).toEqual(['name', 'contains', '50%']);
});

it('should add startsWith filter', () => {
const f = createFilter<{ name: string }>()
.startsWith('name', 'A')
.build();
expect(f).toEqual(['name', 'like', 'A%']);
expect(f).toEqual(['name', 'starts_with', 'A']);
});

it('should add endsWith filter', () => {
const f = createFilter<{ email: string }>()
.endsWith('email', '.com')
.build();
expect(f).toEqual(['email', 'like', '%.com']);
expect(f).toEqual(['email', 'ends_with', '.com']);
});

it('should pass a like() pattern through UNCHANGED — the wildcards are the caller\'s', () => {
// The one method that always meant "pattern", and the one the wire
// lowering broke: `%Industries` matched nothing before #7536.
const f = createFilter<{ name: string }>()
.like('name', '%Industries')
.build();
expect(f).toEqual(['name', 'like', '%Industries']);
});

it('should add ilike filter for a case-insensitive pattern', () => {
const f = createFilter<{ name: string }>()
.ilike('name', '%industries')
.build();
expect(f).toEqual(['name', 'ilike', '%industries']);
});

it('should add exists filter', () => {
Expand Down
Loading
Loading