From bc9cdc1ad35a47c248d547d13d47b94b41ff4d8f Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 06:54:02 +0000 Subject: [PATCH 1/6] docs(skills): collapse removed-key history in the query rules files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit QRY-D-01 — delete the per-key "why it was removed" narration from rules/aggregation.md (`distinct: true`, `array_agg`/`string_agg`, `windowFunctions`) and rules/pagination.md (`cursor`, `distinct`, and the `cursor` Common-Mistakes pair). The keys are tombstoned: `tsc` types them `never` and a query carrying one fails to parse with the prescription, so the only decision-changing half is the replacement — now one table in SKILL.md. QRY-D-02 — delete the alias/push-down bug-history parenthesis at rules/aggregation.md; the behaviour it describes no longer exists. QRY-E-04 — the group-by example is shown in the real `engine.aggregate(obj, …)` shape with legal keys only; `fields` is not in ENGINE_AGGREGATE_OPTION_KEYS and is rejected by name, which the old "readability convention" note did not say. QRY-C-05 — `compareTo` is objectstack-ui's surface; the copy here becomes a pointer. aggregation.md 2226 -> 1846 tok, pagination.md 1381 -> 1158 tok. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- skills/objectstack-query/rules/aggregation.md | 59 ++++++------------- skills/objectstack-query/rules/pagination.md | 36 ++--------- 2 files changed, 23 insertions(+), 72 deletions(-) diff --git a/skills/objectstack-query/rules/aggregation.md b/skills/objectstack-query/rules/aggregation.md index 088a15df72..7b148bbbb0 100644 --- a/skills/objectstack-query/rules/aggregation.md +++ b/skills/objectstack-query/rules/aggregation.md @@ -16,15 +16,8 @@ Guide for building ObjectStack aggregation queries. > ✅ **All six are portable.** `count_distinct` lowers to `COUNT(DISTINCT x)` > on `driver-sql` and turso's remote transport, and `driver-mongodb` / > `driver-memory` compute it too, so the declared-but-uncompiled set is empty. -> The per-aggregation `distinct: true` flag went the other way — **removed in -> 17**, refused at parse. For a deduplicated count, use `count_distinct`. - -> **Removed in 17.** `array_agg` and `string_agg` are no longer part of -> the vocabulary — they were declared and lowered by no SQL backend, so a query -> using them succeeded or failed depending on which driver happened to be -> behind the object. A query carrying either is refused at parse. There is no -> replacement: read the rows with an ordinary `fields` query and shape them in -> the caller, or materialise the roll-up as a stored field. +> `distinct: true`, `array_agg` and `string_agg` are removed keys — see the +> removal table in `SKILL.md`. ## Basic Aggregation @@ -43,20 +36,19 @@ Guide for building ObjectStack aggregation queries. ```typescript // SQL: SELECT region, SUM(amount) AS total, AVG(amount) AS average // FROM sale GROUP BY region -{ - object: 'sale', - fields: ['region'], +const rows = await engine.aggregate('sale', { + groupBy: ['region'], aggregations: [ { function: 'sum', field: 'amount', alias: 'total' }, - { function: 'avg', field: 'amount', alias: 'average' } + { function: 'avg', field: 'amount', alias: 'average' }, ], - groupBy: ['region'] -} +}); ``` -Note: you do NOT need to repeat `groupBy` fields in `fields` — drivers -auto-select every grouped field into the result rows. Listing them in -`fields` (as above) is a readability convention, not a requirement. +Never list the grouped fields in `fields`: drivers auto-select every grouped +field into the result rows, and `fields` is not one of the six keys +`engine.aggregate()` accepts — it is rejected by name (see the calling +convention in `SKILL.md`). ## Date-Bucketed Grouping (dateGranularity) @@ -90,10 +82,7 @@ aggregations (never bucket by hand in app code): - The engine pushes bucketing down to the driver (`DATE_TRUNC` etc.) when the dialect supports that granularity, and transparently falls back to in-memory bucketing otherwise — results are correct either way, **including - the column keys** (earlier SQL drivers ignored `alias`, so an - aliased group came back under the field name when the query was pushed down - and under the alias when it fell back — decided by a capability bit and the - `timezone`, neither of which the caller can see). + the column keys**. ## HAVING Clause @@ -140,8 +129,7 @@ const [row] = await engine.aggregate('user', { > ✅ **`count_distinct` runs everywhere** — `COUNT(DISTINCT field)` on the SQL > faces, the same answer in memory. `field` is REQUIRED; there is no > `COUNT(DISTINCT *)`. The per-aggregation `distinct: true` flag is NOT its -> equivalent: **removed in 17**, refused at parse, because exactly one of the -> six backends that read an aggregation ever honoured it. +> equivalent — it is a removed key. ```typescript // SQL: SELECT COUNT(DISTINCT department) FROM employee @@ -155,16 +143,8 @@ const [row] = await engine.aggregate('user', { ## Window Functions -> ⛔ **REMOVED in `@objectstack/spec` 17 (ADR-0049).** The `QueryAST` -> schema no longer declares `windowFunctions` — the engine never routed the -> property to any driver, so it was silently dropped. The key is tombstoned: -> a query carrying it fails to parse with the upgrade prescription. The one -> live door is the SQL driver's own `findWithWindowFunctions()` (driver-level, -> its own flat input shape; even there the builder drops the `field` argument, -> so `lag(revenue)` renders as `LAG()`). Do not emit `windowFunctions` in -> queries. - -**Working alternatives:** +`windowFunctions` is a removed key (see the removal table in `SKILL.md`). Two +working alternatives: ### Ranking / Top-N per Group @@ -198,13 +178,10 @@ const withTotals = txns.map((t) => ({ ...t, running_total: (runningTotal += t.am ### Period-over-Period -For dashboard widgets, use the higher-level -`compareTo: { kind: 'previousPeriod' | 'previousYear', dimension? }` field on -the widget schema (see *objectstack-ui* → *Period-over-period — `compareTo`*). -The runtime issues the shifted query for you and aligns the result -bucket-for-bucket with the dataset dimension's `dateGranularity`. For ad-hoc comparisons, -run two date-bucketed aggregations (see *Date-Bucketed Grouping* above) -over the two periods and join the buckets in app code. +Dashboard widgets declare it: **objectstack-ui → Period-over-period — +`compareTo`**. For ad-hoc comparisons, run two date-bucketed aggregations (see +*Date-Bucketed Grouping* above) over the two periods and join the buckets in +app code. ## Common Mistakes diff --git a/skills/objectstack-query/rules/pagination.md b/skills/objectstack-query/rules/pagination.md index 9b7f8176d7..cac02fe970 100644 --- a/skills/objectstack-query/rules/pagination.md +++ b/skills/objectstack-query/rules/pagination.md @@ -9,11 +9,8 @@ Guide for implementing pagination in ObjectStack queries. | Offset | UI page navigation, small datasets | Simple, random page access | Slow on large offsets, drift on inserts | | Keyset (manual `where`) | Infinite scroll, real-time feeds | Consistent results, O(1) performance | No random page access | -> ⛔ **The `cursor` query property was REMOVED in `@objectstack/spec` -> 17.** No engine or driver ever read it: a query carrying `cursor` -> silently returned **page 1 forever**. The key is tombstoned — a query -> carrying it fails to parse with the prescription — and -> `QueryBuilder.cursor()` is gone. Implement keyset pagination with a +> ⛔ `cursor` is a removed key (see the removal table in `SKILL.md`), and +> `QueryBuilder.cursor()` is gone with it. Implement keyset pagination with a > `where` filter on the sort key (pattern below). ## Offset Pagination @@ -157,25 +154,6 @@ When building paginated REST endpoints: ## Common Mistakes -### ❌ Wrong: Using the removed `cursor` property - -```typescript -// ❌ cursor was removed in protocol 17 — the tombstone rejects this query outright -{ - object: 'post', - limit: 20, - cursor: { created_at: '2025-01-15T10:30:00Z' } -} - -// ✅ Express the keyset as a where filter on the sort key -{ - object: 'post', - where: { created_at: { $lt: '2025-01-15T10:30:00Z' } }, - orderBy: [{ field: 'created_at', order: 'desc' }], - limit: 20 -} -``` - ### ❌ Wrong: Large offset values ```typescript @@ -215,15 +193,11 @@ When building paginated REST endpoints: ## DISTINCT Queries -> ⛔ **`query.distinct` was REMOVED in `@objectstack/spec` 17.** No driver ever -> rendered `SELECT DISTINCT`. The key is tombstoned — a query carrying it fails -> to parse with the prescription — and `QueryBuilder.distinct()` is gone. Group -> by the fields instead: each unique combination becomes one result row. +> ⛔ `distinct` is a removed key (see the removal table in `SKILL.md`), and +> `QueryBuilder.distinct()` is gone with it. Group by the fields instead: each +> unique combination becomes one result row. ```typescript -// ❌ tombstoned — this query is refused at parse -// { object: 'order', fields: ['customer_id', 'product_category'], distinct: true } - // ✅ groupBy collapses duplicates const rows = await engine.aggregate('order', { groupBy: ['customer_id', 'product_category'], From df1048f1253ccc79de31eb5d0b80fdfb7838f126 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 06:54:14 +0000 Subject: [PATCH 2/6] docs(skills): filters.md keeps the token vocabulary, drops the flow rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit QRY-C-04 — the "a flow filter that loses a condition refuses to run" rule is a flow-node authoring rule, and objectstack-automation already states it. Deleted here; one pointer line stays. (Two things the automation anchor does NOT carry are listed in the PR body as a follow-up for that package.) QRY-F-03 — the string-operator table listed four operators and omitted three that exist: `$icontains`, `$like`, `$ilike`. Added, together with the case rule `packages/spec/src/data/filter.zod.ts` states outright ("`$contains` / `$notContains` / `$startsWith` / `$endsWith` compare CASE-SENSITIVELY. `$icontains` is the case-INSENSITIVE twin"), the ASCII-only folding domain, and the faces that refuse `$like`/`$ilike` rather than approximating them. Funded additions from the delivered objectstack-ui flight, each verified at source before porting — this package is the anchor for the token vocabulary and ui now points here: both `{token}` and `${token}` parse (DATE_MACRO_WRAPPED_RE, shared by the context tokens); `{user_id}` and `{organization_id}` join the near-miss list (CONTEXT_TOKEN_SUGGESTIONS); a token embedded in a larger string is left untouched; `isDateMacroToken` / `isContextToken` from `@objectstack/spec/data` are the author-time check; the build rule is named (`filter-token-unknown`) along with why it exists. Paid in-file by C-04 plus two schema-enforced Common-Mistakes blocks. 2144 -> 2100 tok. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- skills/objectstack-query/rules/filters.md | 103 +++++++++------------- 1 file changed, 41 insertions(+), 62 deletions(-) diff --git a/skills/objectstack-query/rules/filters.md b/skills/objectstack-query/rules/filters.md index 1818f6a700..a70b948f6d 100644 --- a/skills/objectstack-query/rules/filters.md +++ b/skills/objectstack-query/rules/filters.md @@ -19,9 +19,20 @@ Comprehensive guide for building ObjectStack query filters. | String | `$notContains` | `NOT LIKE %?%` | `{ email: { $notContains: 'spam' } }` | | String | `$startsWith` | `LIKE ?%` | `{ code: { $startsWith: 'PRJ-' } }` | | String | `$endsWith` | `LIKE %?` | `{ file: { $endsWith: '.pdf' } }` | +| String | `$icontains` | `LIKE %?%`, case-blind | `{ name: { $icontains: 'john' } }` | +| String | `$like` | `LIKE ?` — caller binds `%` / `_` | `{ code: { $like: 'PRJ-%-26' } }` | +| String | `$ilike` | `$like`, case-blind | `{ code: { $ilike: 'prj-%' } }` | | Null | `$null` | `IS NULL` / `IS NOT NULL` | `{ deleted_at: { $null: true } }` | | Null | `$exists` | `IS NOT NULL` / `IS NULL` | `{ metadata: { $exists: true } }` | +**Case sensitivity is part of the contract** (`filter.zod.ts`): "`$contains` / +`$notContains` / `$startsWith` / `$endsWith` compare CASE-SENSITIVELY. +`$icontains` is the case-INSENSITIVE twin" — ASCII folding only, so `café` does +not match `CAFÉ`. Use `$icontains` for anything a human typed. `$like`/`$ilike` +match the WHOLE value, so a pattern with no wildcard is an exact comparison, not +a substring search; `driver-mongodb`, objectql `having` and service-analytics +refuse them (`INVALID_FILTER`) rather than approximating. + ## Implicit Equality (Shorthand) The most common filter — equality — has a shorthand: @@ -159,31 +170,8 @@ where: { ### ❌ Wrong: Using string operators on non-string fields -```typescript -// ❌ $contains only works on string fields -where: { - age: { $contains: '25' } // age is a number -} - -// ✅ Use comparison operators for numbers -where: { - age: { $eq: 25 } -} -``` - -### ❌ Wrong: Using $between with wrong tuple length - -```typescript -// ❌ $between requires exactly [min, max] -where: { - price: { $between: [10, 50, 100] } -} - -// ✅ Correct: exactly two elements -where: { - price: { $between: [10, 50] } -} -``` +`{ age: { $contains: '25' } }` — the string operators are declared on strings +only; use the comparison operators for numbers and dates. ### ⚠️ Prefer `$null` to a bare `null` comparand @@ -238,6 +226,9 @@ where: { - **Parameterised tokens:** `{N__ago}` / `{N__from_now}` with units `minute|hour|day|week|month|year` — e.g. `{30_days_ago}`, `{2_weeks_from_now}`. +- **Both spellings parse:** `{today}` and `${today}` are the same token + (`DATE_MACRO_WRAPPED_RE` is `/^\$?\{([a-zA-Z0-9_]+)\}$/`, and the context + tokens share it). **Session tokens:** the same value positions accept `{current_user_id}` and `{current_org_id}` (defined in `data/context-tokens.zod.ts`) — the signed-in @@ -247,43 +238,31 @@ user's id and the active organization id. where: { owner: '{current_user_id}', close_date: { $gte: '{current_year_start}' } } ``` -**Scope:** tokens are expanded on **both** sides of the wire, so the same -filter behaves the same wherever it runs — client-side by `resolveDateMacros()` -/ `resolveContextTokens()` in `@object-ui/core`, and server-side by -`resolveFilterTokens()` in `@objectstack/core` (wired into the ObjectQL read -AND write paths — `find`/`findOne`/`count`/`aggregate`/`update`/`delete` — plus -the analytics dataset executor). The driver only ever sees ISO date/timestamp -strings and concrete ids, never `{tokens}`. You may therefore use tokens in a -query issued directly against the engine — and you should: computing "today" at -module load freezes the date into the built artifact. - -This includes a **flow node's** `config.filter`. The flow template engine runs -first there, but it hands a recognised filter placeholder through untouched for -the engine to expand. Flow variables still win — `{record.owner}` resolves as -always, and a flow variable named after a placeholder shadows it. - -**A flow filter that loses a condition refuses to run.** In a filter, a token -that resolves to nothing does not narrow the query — it removes the condition, -which matches *more* rows, and a `delete_record` with every condition gone -means the whole object. So `get_record` / `update_record` / `delete_record` -fail the step, naming the offending template, rather than executing a widened -query. That covers a mistyped field (`{record.ownr}`), an input the run never -received, and a lookup hop (`{record.account.name}` — the trigger record -carries a scalar id; add the relation to the start node's `config.expand`). - -Two of those three are caught earlier: `objectstack validate` **fails** on a -`{record.}` filter token naming an unknown field, or hopping through a -relation the start node does not `expand`, because the runtime has already -committed to refusing that node. The same reference outside a filter — in a -message body, an `http` url, a write payload — stays a warning, since there it -renders a blank rather than widening a query. An unresolved *flow variable* -(`{someInput}`) is not statically checkable and still surfaces at run time. - -**Unknown tokens are rejected, not ignored.** `{current_user}` (the RLS -expression root) and `{this_quarter_start}` are near-misses, not tokens: -`objectstack build` fails on them and the runtime resolver throws. A filter -value that is entirely `{...}` is always read as a placeholder, so a literal -value of that shape is not expressible. +**Scope:** tokens expand on **both** sides of the wire — client-side by +`resolveDateMacros()` / `resolveContextTokens()` in `@object-ui/core`, +server-side by `resolveFilterTokens()` in `@objectstack/core`, wired into the +ObjectQL read AND write paths (`find`/`findOne`/`count`/`aggregate`/`update`/ +`delete`) plus the analytics dataset executor. The driver only ever sees ISO +strings and concrete ids. So use tokens in a hand-issued engine query too: +computing "today" at module load freezes the date into the built artifact. + +A **flow node's** `config.filter` takes these tokens too. What happens when one +of them drops a condition is a flow rule, not a query rule: +**objectstack-automation** — a dropped condition *widens* the query, so +`get_record` / `update_record` / `delete_record` fail the step instead of +running it. + +**Unknown tokens are rejected, not ignored.** A value that is entirely `{...}` +is a placeholder by construction: `objectstack build` fails it (rule +`filter-token-unknown`) and the resolver throws — because an unresolved token +reaches SQL as a **literal**, matches nothing, and renders a widget showing 0, +indistinguishable from "there is no data". Near-misses, not tokens: +`{current_user}` (the RLS expression root), `{this_quarter_start}`, `{user_id}` +(a real `titleFormat` interpolation) and `{organization_id}` (the column name); +the error names the correction. A value that merely *contains* braces is left +untouched — `'user-{current_user_id}'` is a literal. Check a spelling while +authoring with `isDateMacroToken(tok)` / `isContextToken(tok)` from +`@objectstack/spec/data`, passing the token WITHOUT braces. **`*_end` is a calendar DAY.** `{current_year_end}` is `2026-12-31`, so on a `datetime` column `<= {current_year_end}` stops at midnight on the 31st. Use From 3c3a960c44268b075c8c9081fa02c50d593ef123 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 06:54:33 +0000 Subject: [PATCH 3/6] docs(skills): SKILL.md teaches the calling convention it never named MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit QRY-E-03 / QRY-B-04 — the package used two incompatible calling conventions and named neither, and its most-copied example shape (`{ object: 'account', … }` passed as an option bag) is not accepted by any engine method: ENGINE_FIND_OPTION_KEYS and ENGINE_AGGREGATE_OPTION_KEYS are closed sets and an unlisted key is refused by name. The file now opens with the convention — object is the FIRST ARGUMENT, the legal key set per method, and where a bare `{ object, … }` literal IS correct (findData's `query`, an `expand` value). QRY-B-01 — the seven rules stated twice are merged into rules/*: field references, keyset pagination, the OData alias, the aggregation functions table, HAVING, filtered aggregation and window functions. SKILL.md keeps one-line pointers. QRY-D-01 — the four removal-history spans become one 6-row "removed key -> live replacement" table. QRY-C-01 — the search-mirror prescription is objectstack-data's (this file already named data as the anchor); reduced to the rule plus that pointer. QRY-C-03 / QRY-C-05 — the CRM Analytics Query Blueprint and `compareTo` are objectstack-ui's dataset/widget surface; one routing line replaces them. QRY-B-02 / QRY-B-03 / QRY-D-03 / QRY-D-04 — "When to Use This Skill", "Skill Boundaries", the opening blurb, the callout legend and the positioning prose restate the frontmatter or the catalog and change no decision. QRY-E-02 (incidental falsehood 1) — the canonical `expand` example expanded nothing: it projected `fields: ['title','status']` while expanding `assignee` and `project`, dropping both foreign-key columns, and the engine skips a relation whose FK is absent. The example keeps the FK columns and the requirement is now the first Rules bullet. QRY-F-01 — `context` had zero coverage; it is the RLS / system-read escape hatch, and a read without it silently returns fewer rows. Added with the `{ isSystem: true }` example and the query-bag vs trailing-argument rule. QRY-F-02 / QRY-G-03 — the bare-string `search` plus sibling `searchFields` is the canonical Tier-1 form; the three spellings of that one knob are named. QRY-F-03 / QRY-A-01 / QRY-G-02 — the three missing string operators and the case rule; a "which filter dialect" table; one spelling for the version fact. QRY-A-01 also edits the frontmatter description: the list-view filter spec is objectstack-ui's vocabulary, not this package's. 5443 -> 3784 tok. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- skills/objectstack-query/SKILL.md | 555 ++++++++---------------------- 1 file changed, 145 insertions(+), 410 deletions(-) diff --git a/skills/objectstack-query/SKILL.md b/skills/objectstack-query/SKILL.md index 473407c189..dc87fbc8fc 100644 --- a/skills/objectstack-query/SKILL.md +++ b/skills/objectstack-query/SKILL.md @@ -3,90 +3,86 @@ name: objectstack-query description: > Construct ObjectQL queries — filters, sorting, pagination, aggregation, relation expansion, and full-text search. Use when the user is writing a - query DSL expression, picking pagination strategy, or designing a list - view's filter spec. Do not use for defining objects / fields / - relationships (see objectstack-data) or for designing the API endpoint - that exposes a query (see objectstack-api). + query DSL expression or picking a pagination strategy. Do not use for + defining objects / fields / relationships (see objectstack-data), for + designing the API endpoint that exposes a query (see objectstack-api), or + for a list view's filter rules / dashboard datasets (see objectstack-ui). license: Apache-2.0 compatibility: Requires @objectstack/spec 17.x (Zod v4 schemas) metadata: author: objectstack-ai - version: "1.3" + version: "1.4" domain: query tags: query, filter, sort, paginate, aggregate, ObjectQL, full-text --- # Query Design — ObjectStack Query DSL -Expert instructions for constructing data queries using the ObjectStack -Query DSL. This skill covers filter expressions, sorting, pagination, -aggregation, full-text search, and the expand system for related records. +## Calling Convention — `object` is the FIRST ARGUMENT -**Schema vs. runtime:** every callout below says which side a property is on — -⛔ **REMOVED** (tombstoned; a query carrying it fails to parse), ⚠️ **not -enforced** (validates, then silently ignored — never emit it), ✅ **Enforced**. -Each removal callout names the live replacement. +| Surface | Shape | Legal option keys | +|:--|:--|:--| +| engine `find` / `findOne` | `engine.find('task', {…}, { context })` | `context`, `where`, `fields`, `orderBy`, `limit`, `offset`, `search`, `searchFields`, `expand` | +| engine `aggregate` | `engine.aggregate('deal', {…})` | `context`, `where`, `groupBy`, `aggregations`, `having`, `timezone` | +| engine `count` | `engine.count('task', {…})` | `context`, `where` | +| protocol / REST | `findData({ object: 'task', query: {…} })` | `object` sits OUTSIDE the query | +| nested `expand` value | a `QueryAST` — `{ object, fields, where }` | (see **Expand**) | ---- - -## Skill Boundaries - -| Need | Use instead | -|:-----|:------------| -| Define objects, fields, or relationships | **objectstack-data** | -| Define REST API endpoints or auth | **objectstack-api** | -| Build views, dashboards, or apps | **objectstack-ui** | -| Create a plugin or register services | **objectstack-platform** | +`ENGINE_FIND_OPTION_KEYS` / `ENGINE_AGGREGATE_OPTION_KEYS` are **closed sets**: +an unlisted key is refused by name (`find('task') does not recognise option +'bogus'`), never ignored. A standalone `{ object: 'account', limit: 20 }` literal +is therefore a **`QueryAST`** — legal as `findData`'s `query` or an `expand` +value — not an engine option bag. `top` folds to `limit`, `filter` to `where`, +before that check. ---- - -## When to Use This Skill +### Which filter dialect? -- You are constructing a **filter expression** for record retrieval -- You need to **sort or paginate** query results -- You are writing **aggregation queries** (count, sum, avg, group by) -- You need to **expand related records** through lookups -- You are implementing **full-text search** across fields -- You are choosing between **offset vs keyset pagination** +| Writing… | Dialect | Owner | +|:--|:--|:--| +| an ObjectQL `where` | the `$` operators below | this skill | +| a list view / nav-item `filter` | `[{ field, operator, value }]` over the 20-operator `VIEW_FILTER_OPERATORS` enum (`equals`, `icontains`, `is_null`, `before`, `between`, …) — unknown operators are refused at parse | **objectstack-ui** | +| a dataset measure `filter` | the measure's own filter | **objectstack-ui** | ---- +## Execution Context (`context`) -## Core Concepts - -### Query Structure (QueryAST) - -Every ObjectStack query follows the `QuerySchema` structure: +The RLS / system-read escape hatch. A hook, job or endpoint that reads without +one runs as whatever identity the caller carried — org-scoping hooks then return +**fewer rows**, indistinguishable from "there is no data". ```typescript -{ - object: 'account', // Target object (required) - fields: ['name', 'email'], // SELECT — fields to retrieve - where: { status: 'active' }, // WHERE — filter conditions - orderBy: [{ field: 'created_at', order: 'desc' }], // ORDER BY - limit: 20, // LIMIT — max records - offset: 0, // OFFSET — skip records -} +const SYS = { isSystem: true } as const; +const [row] = await engine.find('project', { where: { name }, limit: 1, context: SYS }); ``` -**Key rule:** `object` is the only required property. Everything else is optional. +Pass any SUBSET of the execution envelope (identity, tenant, transaction): +`{ isSystem: true }` for a system read, `{ flowRunId }` for provenance alone. On +the READ methods it may sit in the query bag (above) OR in the trailing options +argument, `engine.find(obj, query, { context })`; the trailing one wins when +both are given. Writes take only the trailing argument. ---- +## Removed Keys → Live Replacement -## Quick Reference — Detailed Rules +| Removed key | Live replacement | +|:--|:--| +| `query.cursor` | keyset paging — `where` on the sort key + `orderBy` + `limit` | +| `query.joins` | `expand`, or a nested relation filter | +| `query.distinct` | `groupBy` the fields — each unique combination is one row | +| `query.windowFunctions` | report/dashboard metadata (**objectstack-ui**), or rank / accumulate in app code | +| aggregation `distinct: true` | `count_distinct` | +| aggregation `array_agg` / `string_agg` | none — read the rows with `fields` and shape them in the caller, or materialise the roll-up as a stored field | -For comprehensive documentation with incorrect/correct examples: +All six are tombstoned in `@objectstack/spec` 17: `tsc` types them `never`, and a +query carrying one fails to parse with the upgrade prescription. The retirement +procedure and the full tombstone register are **objectstack-upgrade**. -- **[Filters](./rules/filters.md)** — All operators, logical combinations, nested relations, date macros -- **[Aggregation](./rules/aggregation.md)** — GroupBy, date bucketing, aggregation functions, driver support -- **[Pagination](./rules/pagination.md)** — Offset vs keyset, best practices, performance +## Quick Reference — Detailed Rules ---- +- **[Filters](./rules/filters.md)** — all operators, logical combinations, nested relations, date macros and session tokens +- **[Aggregation](./rules/aggregation.md)** — groupBy, date bucketing, functions, `having`, per-measure `filter` +- **[Pagination](./rules/pagination.md)** — offset vs keyset, best practices, performance ## Filter Operators -ObjectStack uses a **declarative, database-agnostic** filter DSL inspired by -Prisma, Strapi, and MongoDB. - ### Implicit Equality (Shorthand) The simplest filter — field equals value: @@ -110,9 +106,6 @@ The simplest filter — field equals value: ```typescript { where: { age: { $gte: 18 } } } // SQL: WHERE age >= 18 - -{ where: { created_at: { $gt: '2025-01-01' } } } -// SQL: WHERE created_at > '2025-01-01' ``` ### Set & Range Operators @@ -125,10 +118,7 @@ The simplest filter — field equals value: ```typescript { where: { status: { $in: ['active', 'pending'] } } } -// SQL: WHERE status IN ('active', 'pending') - { where: { amount: { $between: [100, 500] } } } -// SQL: WHERE amount BETWEEN 100 AND 500 ``` ### String Operators @@ -139,12 +129,20 @@ The simplest filter — field equals value: | `$notContains` | Does not contain | `NOT LIKE '%?%'` | | `$startsWith` | Starts with prefix | `LIKE '?%'` | | `$endsWith` | Ends with suffix | `LIKE '%?'` | +| `$icontains` | Contains, case-blind | `LIKE '%?%'` folded | +| `$like` | Whole-value pattern, caller binds `%` / `_` | `LIKE ?` | +| `$ilike` | `$like`, case-blind | `ILIKE ?` | + +`$contains` / `$notContains` / `$startsWith` / `$endsWith` compare +**CASE-SENSITIVELY**; `$icontains` is the case-INSENSITIVE twin (ASCII folding +only). So the user-facing cases want `$icontains`: ```typescript -{ where: { email: { $contains: '@company.com' } } } -// SQL: WHERE email LIKE '%@company.com%' +{ where: { email: { $icontains: '@company.com' } } } ``` +Full table, `$like` portability and the `$ilike` boundary: **[filter rules](./rules/filters.md)**. + ### Null & Existence Operators | Operator | Purpose | SQL / NoSQL | @@ -154,7 +152,6 @@ The simplest filter — field equals value: ```typescript { where: { deleted_at: { $null: true } } } -// SQL: WHERE deleted_at IS NULL ``` ### Logical Operators @@ -163,34 +160,20 @@ Combine conditions with `$and`, `$or`, and `$not`: ```typescript // OR: active accounts OR accounts with high revenue -{ - where: { - $or: [ - { status: 'active' }, - { revenue: { $gt: 1000000 } } - ] - } -} +{ where: { $or: [{ status: 'active' }, { revenue: { $gt: 1000000 } }] } } // AND + OR combined { where: { $and: [ { type: 'enterprise' }, - { $or: [ - { region: 'us' }, - { region: 'eu' } - ]} + { $or: [{ region: 'us' }, { region: 'eu' }] }, ] } } // NOT: exclude closed accounts -{ - where: { - $not: { status: 'closed' } - } -} +{ where: { $not: { status: 'closed' } } } ``` ### Nested Relation Filters @@ -198,39 +181,14 @@ Combine conditions with `$and`, `$or`, and `$not`: Filter through relationships without an explicit join: ```typescript -// Filter accounts where the related contact has a verified profile -{ - object: 'account', - where: { - contact: { // Relation field name - profile: { // Nested relation - verified: true - } - } - } -} +// Accounts whose related contact has a verified profile +{ object: 'account', where: { contact: { profile: { verified: true } } } } ``` -### Field References (Cross-Field Comparisons) +### Cross-field comparisons -> ✅ **Enforced.** A `{ $field: '...' }` comparand compares two columns of the -> same row. The in-memory evaluator resolves the reference against the record; -> `driver-sql` pushes it down as a column-to-column predicate. Same rows. - -```typescript -// ✅ Accounts whose actual revenue beat the estimate -{ - where: { - actual_revenue: { $gt: { $field: 'estimated_revenue' } } - } -} -``` - -Legal in a **comparison** position only. As an `$in` / `$nin` member or a -`$between` endpoint it is refused at parse — no evaluation path resolves a -reference there. - ---- +`{ $field: '...' }` compares two columns of the same row, in a **comparison** +position only — see **[filter rules → Field References](./rules/filters.md)**. ## Sorting @@ -251,180 +209,70 @@ Sort with `orderBy` — an array of sort nodes: - Default `order` is `'asc'` — you can omit it for ascending sorts - Sort fields should be indexed for performance (see **objectstack-data** indexing rules) ---- - ## Pagination -### Offset Pagination (Simple) - -```typescript -{ - object: 'account', - limit: 20, - offset: 40, // Skip first 40 records (page 3) -} -``` - -**When to use:** UI pages, small datasets (<100K records), when you need "jump to page N". - -**Pitfall:** Offset pagination degrades on large offsets — the database still scans skipped rows. - -### Keyset Pagination (Performant) - -> ⛔ **`query.cursor` was REMOVED in `@objectstack/spec` 17.** No -> engine or driver ever read it — a query carrying `cursor` silently returned -> **page 1 forever**. The key is tombstoned (a query carrying it fails to -> parse with the prescription) and `QueryBuilder.cursor()` is gone. Do keyset -> pagination with `where` + `orderBy` + `limit`: - ```typescript -// First page -{ - object: 'account', - orderBy: [{ field: 'created_at', order: 'desc' }], - limit: 20, -} - -// Next page — filter past the last record you've seen -{ - object: 'account', - where: { created_at: { $lt: lastSeenCreatedAt } }, - orderBy: [{ field: 'created_at', order: 'desc' }], - limit: 20, -} +// Offset paging — page 3 +{ object: 'account', limit: 20, offset: 40 } ``` -**When to use:** Infinite scroll, APIs, large datasets, real-time feeds. - -**Rule:** The keyset `where` field must match the `orderBy` field (use a -unique or near-unique column such as `created_at` or `id`) so -`WHERE created_at < ?` picks up exactly where the previous page ended. - -### OData Compatibility - -`top` is an alias for `limit` (for OData-style APIs): - -```typescript -{ object: 'account', top: 50 } -// Equivalent to: { object: 'account', limit: 50 } -``` +**When to use:** UI pages, small datasets, "jump to page N". It degrades on +large offsets — the database still scans the skipped rows. ---- +For **keyset** paging (infinite scroll, APIs, large datasets, real-time feeds), +filter past the last row you saw with a `where` on the sort key, and always +`orderBy` that same field in that same direction — the pattern, the direction +rule and the pitfalls are **[pagination rules](./rules/pagination.md)**. ## Aggregation -### Basic Aggregation Functions - -| Function | Purpose | SQL | -|:---------|:--------|:----| -| `count` | Count rows | `COUNT(*)` or `COUNT(field)` | -| `sum` | Sum values | `SUM(field)` | -| `avg` | Average | `AVG(field)` | -| `min` | Minimum | `MIN(field)` | -| `max` | Maximum | `MAX(field)` | -| `count_distinct` | Unique count | `COUNT(DISTINCT field)` | - -> ✅ **All six are portable.** `count_distinct` lowers to `COUNT(DISTINCT x)` -> on every SQL face and computes identically on the in-memory path, so the -> declared-but-uncompiled set is empty. The per-aggregation `distinct: true` -> flag went the other way — **removed in 17**, refused at parse; the live -> spelling for a deduplicated count is `count_distinct`. - -> **Removed in 17.** `array_agg` and `string_agg` left this vocabulary: -> declared but lowered by no SQL backend, so whether they worked depended on -> which driver sat behind the object. Either one is now refused at parse. There -> is no replacement — read the rows with an ordinary `fields` query and shape -> them in the caller, or materialise the roll-up as a stored field. - -### GroupBy + Aggregation +The six functions (`count`, `sum`, `avg`, `min`, `max`, `count_distinct`), date +bucketing, `having`, and the per-measure `filter` are +**[aggregation rules](./rules/aggregation.md)**. The call shape: ```typescript // Total revenue per region -{ - object: 'deal', - fields: ['region'], - aggregations: [ - { function: 'sum', field: 'amount', alias: 'total_revenue' }, - { function: 'count', alias: 'deal_count' }, - ], - groupBy: ['region'], - orderBy: [{ field: 'total_revenue', order: 'desc' }], -} -// SQL: SELECT region, SUM(amount) AS total_revenue, COUNT(*) AS deal_count -// FROM deal GROUP BY region ORDER BY total_revenue DESC -``` - -`groupBy` entries can also be structured objects for **date bucketing** — -`{ field: 'closed_at', dateGranularity: 'quarter' }` — see -[Aggregation rules](./rules/aggregation.md) for the full pattern. - -### HAVING Clause - -> ✅ **Enforced.** The engine applies `having` AFTER aggregation, -> on both the native-driver path and the in-memory fallback. It references -> the **aggregated row's columns** — aggregation aliases and groupBy -> projections — with the ordinary FilterCondition operators plus -> `$and`/`$or`/`$not`. An unknown operator is rejected loudly, never ignored. - -```typescript -// ✅ Only regions with more than 100k revenue const rows = await engine.aggregate('deal', { groupBy: ['region'], aggregations: [ { function: 'sum', field: 'amount', alias: 'total_revenue' }, + { function: 'count', alias: 'deal_count' }, ], - having: { total_revenue: { $gt: 100000 } }, }); +// SQL: SELECT region, SUM(amount) AS total_revenue, COUNT(*) AS deal_count +// FROM deal GROUP BY region ``` -### Filtered Aggregation - -> ✅ **Enforced.** A per-aggregation `filter` scopes that one measure, so a -> total and a conditional count share one call. Any aggregation carrying a -> non-empty `filter` forces the in-memory path — no driver compiles a -> conditional aggregate, and one reached directly refuses `NOT_IMPLEMENTED`; -> unfiltered aggregations keep native push-down. - -```typescript -// ✅ Total and conditional counts in ONE call -const [kpis] = await engine.aggregate('order', { - aggregations: [ - { function: 'count', alias: 'total_orders' }, - { function: 'count', alias: 'high_value_orders', - filter: { amount: { $gt: 1000 } } }, - ], -}); -// An unknown operator inside `filter` refuses INVALID_FILTER/400 — it never -// silently answers the unfiltered number. -``` - ---- +`fields` and `orderBy` are NOT in `ENGINE_AGGREGATE_OPTION_KEYS` — do not put +them in an `aggregate` bag. Grouped fields are auto-selected into the result +rows; read each measure under its `alias`, and reference that same name from +`having`. `groupBy` entries may be objects for date bucketing — +`{ field: 'closed_at', dateGranularity: 'quarter' }`. ## Expand (Related Records) -Load related records through lookup/master_detail fields: +Load related records through lookup / master_detail fields. **Keep the foreign +key in `fields`** — the relation is carried by that column: ```typescript -{ - object: 'task', - fields: ['title', 'status'], +const tasks = await engine.find('task', { + fields: ['title', 'status', 'assignee', 'project'], // the FK columns stay expand: { - assignee: { - object: 'user', - fields: ['name', 'email'], - }, + assignee: { object: 'user', fields: ['name', 'email'] }, project: { object: 'project', fields: ['name'], - expand: { - org: { object: 'org', fields: ['name'] } // Nested expand - } - } - } -} + expand: { org: { object: 'org', fields: ['name'] } }, // nested expand + }, + }, +}); ``` **Rules:** +- **The projection must RETAIN the foreign-key column.** `fields: ['title']` + with `expand: { project: … }` resolves **nothing**: the engine reads the FK + off each record and skips the relation when it is absent, so the call returns + rows with no related data and no error. - Max expand depth is **3** by default - The engine resolves expands via batch `$in` queries (not N+1) - Keys in `expand` must be lookup or master_detail field names @@ -433,52 +281,16 @@ Load related records through lookup/master_detail fields: `orderBy` are NOT applied on this path. To paginate or sort related records, query the related object directly. ---- - -## Joins - -> ⛔ **REMOVED in `@objectstack/spec` 17 (ADR-0049).** `query.joins` -> (and the `JoinNode`/`JoinType`/`JoinStrategy` vocabulary) is gone from the -> `QueryAST` schema — no engine or driver ever consumed it, so it only ever -> declared a capability that did not run. The key is tombstoned: authoring it -> is a `tsc` error, and a query carrying it (even `joins: []`) fails to parse -> with the upgrade prescription. Do not emit `joins`. - -**Working alternatives** (both implemented): -- **`expand`** — load related records through lookup / master_detail fields - (see previous section). -- **Nested relation filters** — filter a parent by conditions on a related - object without an explicit join: - -```typescript -// Orders whose customer is in the US — no join needed -{ - object: 'order', - fields: ['id', 'amount'], - where: { customer: { country: 'US' } }, -} -``` - ---- - ## Full-Text Search -Only the **`query` + `fields`** subset of the search schema executes. The -engine expands the search string into a driver-agnostic filter: each term -becomes an `$or` of `$icontains` predicates (the case-INSENSITIVE twin of the -case-sensitive `$contains`) across the resolved searchable fields, and multiple -whitespace-separated terms are **AND-ed** (every term must hit some field). -`select`/`status` fields match by option *label*, mapped to stored values. +The canonical form is a **bare string** with a sibling `searchFields`: ```typescript -{ - object: 'article', - search: { - query: 'machine learning', - fields: ['title', 'content'], - }, +const rows = await engine.find('article', { + search: 'machine learning', + searchFields: ['title', 'content'], limit: 10, -} +}); // Executes as: // { $and: [ // { $or: [{ title: { $icontains: 'machine' } }, { content: { $icontains: 'machine' } }] }, @@ -486,85 +298,32 @@ whitespace-separated terms are **AND-ed** (every term must hit some field). // ]} ``` -Omit `fields` to search the object's declared `searchableFields` (or an -auto-default of name/title + short-text fields), resolved server-side. - -`fields` can only **narrow** that set, never widen it: over the REST/protocol -ingress a name outside it is `400 INVALID_FIELD`, not a silent -fall-back to the full scan. - -### ⛔ Searching by a related record's title — mirror the value, always - -`search` scans **the queried object's own columns**. A dotted path is never a -search target: unlike `fields` (projection) / `sort` / `filters`, the search axis -does not resolve traversal, so `searchFields: ['project_id.name']` is **refused**: - -```text -Unknown field 'project_id.name' on object 'task'. '$searchFields' narrows which -columns 'search' scans, so a name the object does not declare cannot narrow -anything — and the engine used to drop it and scan the default columns instead, -answering a NARROWER search with a WIDER one. 'search' scans this object's own -columns; a related record's column cannot be a search target. -``` - -**This is the one prescription — emit it every time.** Copy the related record's -title into a **stored field on the queried object** and search that field. A task -list searched by project name gets a `project_name` text column on `task`, -maintained on write and listed in `task.searchableFields`: - -```typescript -{ - object: 'task', - search: { query: 'apollo', fields: ['name', 'project_name'] }, - limit: 20, -} -// Expands to a single-table scan — no traversal, every driver: -// { $or: [ -// { name: { $icontains: 'apollo' } }, -// { project_name: { $icontains: 'apollo' } }, -// ]} -``` - -❌ The mirror must be a **stored** field — a `formula` field is virtual, no -driver materializes a column for it, so a search predicate against one has -nothing to scan. Two guards catch that: lint errors on a virtual -`searchableFields` entry, and the ingress gate refuses one by name. The -trade-off is mirror maintenance — hooks on both write paths (child re-parented, -parent renamed) plus a backfill for rows written around the hooks. - -Cross-object search paths are rejected by design, not pending. Modelling side of -this (the field, the hooks, the lint wording): **objectstack-data → Search Fields -(`searchableFields`)**. To *filter* by a related record's column — a different -axis — use a [nested relation filter](#nested-relation-filters); to *display* it, -use [`expand`](#expand-related-records). - -> ⚠️ **`[EXPERIMENTAL — not enforced]`:** `fuzzy`, `boost`, -> `operator`, `minScore`, `language`, and `highlight` validate against the -> schema but are never read — their `.describe()` markers now say so. Terms -> are always AND-ed; there is no relevance scoring or highlighting. - ---- - -## Window Functions (Analytics) - -> ⛔ **REMOVED from the request surface in `@objectstack/spec` 17.** -> `query.windowFunctions` is gone from the `QueryAST` schema — the engine -> never routed it to any driver, so every OVER clause it declared was -> silently dropped. The key is tombstoned (a query carrying it fails to -> parse with the prescription), and the `WindowFunction`/`WindowSpec`/ -> `WindowFunctionNode` exports left with it. Do not emit `windowFunctions`. -> The one live door is the SQL driver's own `findWithWindowFunctions()` -> method (driver-level, its own flat input shape — and even there the -> builder drops the `field` argument, so `lag(revenue)` renders as `LAG()`). - -**Working alternatives:** -- **Ranking / top-N per group and running totals:** model them in - report/dashboard metadata (groupings, measures, `dateGranularity` - bucketing, `compareTo` for period-over-period) — see **objectstack-ui**. -- **Ad-hoc analysis:** fetch the ordered rows (`orderBy` + `limit`) and - compute ranks or running sums in application code. - ---- +Each term becomes an `$or` of `$icontains` predicates across the resolved +searchable fields, and whitespace-separated terms are **AND-ed** (every term +must hit some field). `select`/`status` fields match by option *label*, mapped +to stored values. + +**One knob, three spellings:** emit `searchFields` (the engine option). The +protocol normalizes `$searchFields` onto it, and the object form +`search: { query, fields }` spells the same narrowing `fields`. + +Omit it to search the object's declared `searchableFields` (or an auto-default +of name/title + short-text fields), resolved server-side. It can only **narrow** +that set, never widen it: over the REST/protocol ingress a name outside it is +`400 INVALID_FIELD`, not a silent fall-back to a full scan. The object form +`search: { query, fields }` stays available for the Tier-2 knobs below. + +> ⚠️ **Validates, then silently ignored — never emit these.** `fuzzy`, `boost`, +> `operator`, `minScore`, `language` and `highlight` are the whole set; their +> `.describe()` markers say so. Terms are always AND-ed; there is no relevance +> scoring or highlighting. + +**`search` never traverses.** A dotted path is refused — +`searchFields: ['project_id.name']` names a column `task` does not declare. +Mirror the related record's title into a **stored** field on the queried object +and search that; the field, the write hooks and the lint wording are +**objectstack-data → Search Fields (`searchableFields`)**. To *filter* by a +related record's column use a nested relation filter; to *display* it, `expand`. ## Common Patterns @@ -574,63 +333,42 @@ use [`expand`](#expand-related-records). |:---------|:----| | Load lookup fields for display | `expand` | | Filter parent by child conditions | Nested relation filter | -| **Keyword-search by a related record's title** | **Mirror the title into a stored field on this object and search that** — `search` never traverses (see **Full-Text Search** above) | -| Simple parent→child navigation | `expand` | +| **Keyword-search by a related record's title** | **Mirror the title into a stored field on this object and search that** — `search` never traverses | | Paginate/sort a parent's related records | Query the related object directly | -| Analytical queries across objects | Report/dashboard metadata, or separate queries combined in app code (`joins` was removed — see above) | +| Analytical queries across objects | Report/dashboard metadata, or separate queries combined in app code | ### Pagination Pattern for APIs ```typescript -// Page-based API response -{ - object: 'account', +const page = await engine.find('account', { where: { status: 'active' }, fields: ['id', 'name', 'email'], orderBy: [{ field: 'name', order: 'asc' }], limit: 20, - offset: (page - 1) * 20, -} + offset: (pageNumber - 1) * 20, +}); ``` ### Dashboard Aggregation Pattern -Every KPI on a dashboard shares **one** aggregate call — unconditional -measures plain, conditional ones carrying their own `filter`. `where` scopes -the whole call, so reach for it only when every measure wants the same scope: +Every KPI on a dashboard shares **one** aggregate call — unconditional measures +plain, conditional ones carrying their own `filter`. `where` scopes the whole +call, so reach for it only when every measure wants the same scope: ```typescript -// KPI dashboard: one call, conditional measures scoped per aggregation const [kpis] = await engine.aggregate('deal', { aggregations: [ { function: 'count', alias: 'total_deals' }, { function: 'sum', field: 'amount', alias: 'pipeline_value' }, { function: 'avg', field: 'amount', alias: 'avg_deal_size' }, - { function: 'count', alias: 'won_deals', - filter: { stage: 'closed_won' } }, + { function: 'count', alias: 'won_deals', filter: { stage: 'closed_won' } }, ], }); ``` ---- - -## CRM Analytics Query Blueprint - -Model analytics in dashboard/report metadata rather than hand-written query -code — the renderer issues the queries for you: - -| Query Need | Pattern | -|:--|:--| -| KPI widgets | Aggregates (`sum`, `count`, `avg`) over the object, each conditional KPI scoped by the widget/dataset filter. Add `compareTo: { kind: 'previousPeriod' \| 'previousYear' }` on the widget for a one-line period-over-period delta (the bare string form was removed in 17). | -| Time-series chart | Date filters + `dateGranularity: 'day' \| 'week' \| 'month' \| 'quarter' \| 'year'` on the widget's dataset selection for server-side bucketing — never bucket by hand on the client. Pair with `compareTo` for an aligned YoY overlay. | -| Matrix report | Dataset-bound `rows` (down) + `columns` (across) + a `dateGranularity` dimension | -| Funnel summary | Multi-level grouping (`owner -> stage`) + aggregated measures | -| Operational filter | Prefer declarative operators (`$ne`, `$nin`, `$gte`) over hardcoded SQL | - -For metadata app development, model analytics in report/dashboard metadata first; -only fall back to custom query code when schema limits require it. - ---- +Dashboards and reports themselves — KPI widgets, `compareTo`, `dateGranularity` +bucketing, matrix rows/columns — are metadata, not hand-written queries: model +them in **objectstack-ui** and the renderer issues the queries. ## Verify your work @@ -647,12 +385,9 @@ A dashboard widget whose `dataset` / `dimensions` / `values` don't resolve fails here instead of rendering an empty chart (ADR-0021). In a scaffolded project the gate is `npm run validate`. See objectstack-platform → **Verify your work**. ---- - ## References See [references/_index.md](./references/_index.md) for the full list of Zod schemas (with one-line descriptions) — pointers into `node_modules/@objectstack/spec/src/`. Always `Read` the source for exact field shapes; do not rely on memory of property names. - From 2b0967d48ce00aad5e40bacce049d5ac83d784b5 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 06:54:34 +0000 Subject: [PATCH 4/6] docs(skills): regenerate the skills reference after the query description edit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generator output only — `pnpm --filter @objectstack/spec gen:skill-docs`. `skills/README.md` is byte-unchanged: it renders only the description's FIRST sentence, which QRY-A-01 did not touch. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- content/docs/ai/skills-reference.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/ai/skills-reference.mdx b/content/docs/ai/skills-reference.mdx index 657b24c9bb..680f58c192 100644 --- a/content/docs/ai/skills-reference.mdx +++ b/content/docs/ai/skills-reference.mdx @@ -91,9 +91,9 @@ Do not use for querying data (see objectstack-query) or for plugin / kernel hook Construct ObjectQL queries — filters, sorting, pagination, aggregation, relation expansion, and full-text search. -Use when the user is writing a query DSL expression, picking pagination strategy, or designing a list view's filter spec. +Use when the user is writing a query DSL expression or picking a pagination strategy. -Do not use for defining objects / fields / relationships (see objectstack-data) or for designing the API endpoint that exposes a query (see objectstack-api). +Do not use for defining objects / fields / relationships (see objectstack-data), for designing the API endpoint that exposes a query (see objectstack-api), or for a list view's filter rules / dashboard datasets (see objectstack-ui). **Tags:** `query`, `filter`, `sort`, `paginate`, `aggregate`, `ObjectQL`, `full-text` From 96635262aee9bc4d3645fc09c3cd243f79218ed6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 07:20:43 +0000 Subject: [PATCH 5/6] =?UTF-8?q?docs(skills):=20contract=20review=20round?= =?UTF-8?q?=201=20=E2=80=94=20the=20passthrough=20keys,=20and=20the=20$lik?= =?UTF-8?q?e=20faces?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two spans, both re-measured at source before touching them. 1. The calling-convention table listed nine keys for engine `find`/`findOne` and called the set closed. `packages/objectql/src/engine.ts:396-400` spreads `ENGINE_DRIVER_PASSTHROUGH_KEYS` (`:368-370`: `transaction`, `tenantId`, `tenantIds`, `timezone`, `bypassTenantAudit`, `preserveAudit`) into it, so as written the table said an explicit `tenantId` is refused, which is false. The row now names all fifteen, and the prose carries the reason the source gives at `:360-366`: on find/findOne (and update/delete) the option bag IS the base of the driver options, while `count`/`aggregate` never forward the bag, so those keys are deliberately illegal there. `count` and `aggregate` rows unchanged — both were already exact. 2. Review asked me to verify the `$like`/`$ilike` face list at source or drop it. Verified; all three refusals hold, and each is an ALLOWLIST MISS, which is why grepping the drivers for `$like` does not find them: - driver-mongodb — `mongodb-filter.ts:1071` throws `Unsupported filter operator "$like" …`; its `unsupportedFilterError` (`:366-371`) sets `INVALID_FILTER` / 400. Pinned at `mongodb-operator-key-clobber.test.ts:158-161`. - objectql `having` — `having-filter.ts:136-141` `CONDITION_OPERATORS` is sixteen operators without `$like`/`$ilike`; anything else reaches `unknownOperator` (`:174-205`) and `invalidFilterError`, `INVALID_FILTER` / 400 (the table at `:157-158` names the faces sharing that envelope). - service-analytics — `strategies/filter-normalizer.ts:418-435` `MONGO_TO_CUBE_OP` omits them; the miss branch at `:1032-1046` throws `invalidFilterError` (`:403-406`, `INVALID_FILTER` / 400) with `Unsupported filter operator "$like" …`. The review's own two readings were the ones that did not survive measurement, so the sentence is corrected rather than weakened: `driver-memory` ANSWERS these operators (`memory-driver.ts:58,1522`, `memory-matcher.ts:458`), and `filter-refusal.ts:720-726` refuses only a DANGLING ESCAPE in the pattern via the spec's shared `hasDanglingLikeEscape`. The prose now states which faces answer, which refuse, and the escape rule that binds all of them. Paid in-file: `rules/filters.md` 2100 -> 2136 (ceiling 2149, and still under the 2144 it started at) by collapsing two Common-Mistakes blocks that restated the Logical Operators and `$null` sections above them. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- skills/objectstack-query/SKILL.md | 10 ++++-- skills/objectstack-query/rules/filters.md | 43 +++++++---------------- 2 files changed, 20 insertions(+), 33 deletions(-) diff --git a/skills/objectstack-query/SKILL.md b/skills/objectstack-query/SKILL.md index dc87fbc8fc..d2b039564b 100644 --- a/skills/objectstack-query/SKILL.md +++ b/skills/objectstack-query/SKILL.md @@ -22,19 +22,25 @@ metadata: | Surface | Shape | Legal option keys | |:--|:--|:--| -| engine `find` / `findOne` | `engine.find('task', {…}, { context })` | `context`, `where`, `fields`, `orderBy`, `limit`, `offset`, `search`, `searchFields`, `expand` | +| engine `find` / `findOne` | `engine.find('task', {…}, { context })` | `context`, `where`, `fields`, `orderBy`, `limit`, `offset`, `search`, `searchFields`, `expand` — **plus** the six driver passthrough keys `transaction`, `tenantId`, `tenantIds`, `timezone`, `bypassTenantAudit`, `preserveAudit` | | engine `aggregate` | `engine.aggregate('deal', {…})` | `context`, `where`, `groupBy`, `aggregations`, `having`, `timezone` | | engine `count` | `engine.count('task', {…})` | `context`, `where` | | protocol / REST | `findData({ object: 'task', query: {…} })` | `object` sits OUTSIDE the query | | nested `expand` value | a `QueryAST` — `{ object, fields, where }` | (see **Expand**) | `ENGINE_FIND_OPTION_KEYS` / `ENGINE_AGGREGATE_OPTION_KEYS` are **closed sets**: -an unlisted key is refused by name (`find('task') does not recognise option +a key outside the row is refused by name (`find('task') does not recognise option 'bogus'`), never ignored. A standalone `{ object: 'account', limit: 20 }` literal is therefore a **`QueryAST`** — legal as `findData`'s `query` or an `expand` value — not an engine option bag. `top` folds to `limit`, `filter` to `where`, before that check. +The passthrough six ride along on `find`/`findOne` (and on `update`/`delete`) +because there the option bag IS the base of the driver options, which is how an +explicit `tenantId` reaches the driver. `count` and `aggregate` never forward the +bag, so on those two the same keys are deliberately ILLEGAL — accepting them +would be the silently-ignored option this check exists to close. + ### Which filter dialect? | Writing… | Dialect | Owner | diff --git a/skills/objectstack-query/rules/filters.md b/skills/objectstack-query/rules/filters.md index a70b948f6d..ab88d503d2 100644 --- a/skills/objectstack-query/rules/filters.md +++ b/skills/objectstack-query/rules/filters.md @@ -30,8 +30,12 @@ Comprehensive guide for building ObjectStack query filters. `$icontains` is the case-INSENSITIVE twin" — ASCII folding only, so `café` does not match `CAFÉ`. Use `$icontains` for anything a human typed. `$like`/`$ilike` match the WHOLE value, so a pattern with no wildcard is an exact comparison, not -a substring search; `driver-mongodb`, objectql `having` and service-analytics -refuse them (`INVALID_FILTER`) rather than approximating. +a substring search. Their reach is narrower than the rest of the table: the SQL +family, `driver-memory` and `@objectstack/formula` answer them, while +`driver-mongodb`, objectql `having` and service-analytics each keep an operator +allowlist that omits them and refuse — `Unsupported filter operator`, +`INVALID_FILTER` / 400 — rather than approximating. A pattern ending in a lone +unpaired backslash is refused by every face. ## Implicit Equality (Shorthand) @@ -149,24 +153,10 @@ where: { ## Common Mistakes -### ❌ Wrong: Multiple operators on different fields inside $or +### ❌ Wrong: expecting sibling keys to be an OR -```typescript -// ❌ This is an AND, not an OR -where: { - role: 'admin', - status: 'active' -} -// Correct only if you want both conditions - -// ✅ For OR, wrap in $or array -where: { - $or: [ - { role: 'admin' }, - { status: 'active' } - ] -} -``` +`where: { role: 'admin', status: 'active' }` is an AND — sibling keys always +are. For OR, wrap them in a `$or` array (see **Logical Operators** above). ### ❌ Wrong: Using string operators on non-string fields @@ -175,18 +165,9 @@ only; use the comparison operators for numbers and dates. ### ⚠️ Prefer `$null` to a bare `null` comparand -```typescript -// ⚠️ Works — a bare null lowers to IS NULL on both paths — but it reads -// as "equals null" and has no IS NOT NULL spelling -where: { - deleted_at: null -} - -// ✅ Explicit, and `$null: false` is IS NOT NULL -where: { - deleted_at: { $null: true } -} -``` +`where: { deleted_at: null }` works — a bare null lowers to `IS NULL` on both +paths — but it reads as "equals null" and has no `IS NOT NULL` spelling. Write +`{ deleted_at: { $null: true } }` instead; `$null: false` is `IS NOT NULL`. ## Date Filtering Patterns From 8208cf30b56487b97e2de011cc5b7ded92346d28 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 2 Sep 2026 07:56:55 +0000 Subject: [PATCH 6/6] chore(gates): ratchet the role-word baseline down for rules/filters.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `check:role-word` reds on IMPROVEMENT as well as regression: round 2's deletion of two Common-Mistakes blocks in `skills/objectstack-query/rules/filters.md` removed one baselined occurrence, so the gate reported role-word count improved 9 -> 8 — ratchet DOWN: run `node scripts/check-role-word.mjs --update` and commit the baseline. This is that commit, and nothing else: `node scripts/check-role-word.mjs --update` moved exactly one row (9 -> 8) and no other, verified by `git diff` before committing. The gate is green after it. Why it reached CI rather than my machine: round 2 re-ran only the five card-named gates instead of re-deriving the union, and this family is pulled in by the prose paths, not by the card's list. Round 3 runs the whole re-derived list — which this baseline file itself widens. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1 --- scripts/role-word-baseline.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/role-word-baseline.json b/scripts/role-word-baseline.json index 119885f46c..76ceee2206 100644 --- a/scripts/role-word-baseline.json +++ b/scripts/role-word-baseline.json @@ -39,7 +39,7 @@ "skills/objectstack-data/SKILL.md": 4, "skills/objectstack-data/rules/relationships.md": 1, "skills/objectstack-platform/SKILL.md": 2, - "skills/objectstack-query/rules/filters.md": 9, + "skills/objectstack-query/rules/filters.md": 8, "skills/objectstack-ui/SKILL.md": 2, "skills/objectstack-upgrade/SKILL.md": 1 }