Skip to content

feat(objectql): engine option bags are a closed contract — unknown keys throw instead of silently doing nothing - #4400

Merged
os-zhuang merged 2 commits into
mainfrom
claude/engine-unknown-key-enforcement-b44637
Jul 31, 2026
Merged

feat(objectql): engine option bags are a closed contract — unknown keys throw instead of silently doing nothing#4400
os-zhuang merged 2 commits into
mainfrom
claude/engine-unknown-key-enforcement-b44637

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes#4371 — option (2), the complete answer (option (1) landed in #4387). Scoped by a full caller survey of every engine call site in this repo + cloud, plus every key the engine itself reads.

What changes

All six engine methods reject non-null option keys they do not execute, naming the legal set. Retired keys (cursor/distinct) quote their #4286 tombstone. null stays a withdrawal, same as the fold's rule.

Per-method legal keys = the method's Engine*OptionsSchema keys plus the documented extras the survey proved live:

A drift pin asserts each set against its schema's .shape (minus tombstones, plus the extras), so a key added to the spec cannot be silently rejected.

What the survey turned up and this PR also closes

  1. Bag object overrode the resolved object on the AST{ object, ...query } spread order let query.object win, splitting ast.object from the table actually queried (drivers use the first argument; middleware and hooks reading ast.object were lied to). AST now keeps the resolved name; a direct call passing object is rejected; the protocol refuses a POST-body object contradicting the route (400 QUERY_OBJECT_MISMATCH) instead of picking a winner.
  2. findData leaked protocol vocabulary onto the engine bag (object, count, joins, windowFunctions, cursor, distinct, non-aggregate having) — now stripped at the boundary; wire behaviour unchanged.
  3. Nested expand ASTs (expand: { rel: { sort } }) reject the four wire-only spellings exactly like the top-level bag did in fix(objectql): reject wire-only alias spellings (sort/select/skip/populate) on direct engine calls #4387.
  4. $search/$searchFields engine reads removed — the protocol normalizes to bare keys; a direct call passing the OData spelling now throws instead of half-working on find only.
  5. DbQueueAdapter.purge/purgeFailed were broken on the real engine: both passed { id }, a key the engine never read — purge deleted nothing (every delete threw "Delete requires an ID" into a warn-level catch) and purgeFailed always threw. The test fake's delete implemented the { id } signature the real engine rejects, which is exactly how this stayed green. Both fixed to { where: { id } }; the fake now matches the real contract.
  6. 19 test bags carrying the dead filters key cleaned up — two were intent-carrying (a locale multi-update and an oidc dogfood assert that both silently scanned unfiltered).
  7. driver.find JSDoc example showed wire spellings a driver can never receive — now canonical.

Tests

  • 21 pins in engine-unknown-option.test.ts (per-method rejection, tombstones, null withdrawal, extras legal where they work and rejected where they don't, object rejection, nested-expand rejection + canonical expand works, drift pin).
  • 3 pins in protocol-data.test.ts (protocol strips its keys; matching body object tolerated; contradicting body object 400s before the engine).
  • Full local runs: objectql 1486, runtime 1001, rest 539, security 677, auth 579, automation 538, cli 629, settings 196, client 204, datasource 180, approvals 330, sharing 226, metadata-protocol 136, queue 20 — all green; spec generated-artifact gates green (docs regenerated); typecheck green.

Wire/HTTP callers are unaffected. Direct engine callers passing junk keys get an error that names the fix.

🤖 Generated with Claude Code

os-zhuangand others added 2 commits July 31, 2026 18:44
…re spellings
The @example taught filters/sort/top — keys a driver can never receive:
the engine folds/rejects wire spellings before this layer (#4346, #4371),
and QuerySchema declares where/orderBy/limit. Also dedupes the doubled
@returns tag. Comment-only; no generated artifact reads @example (all 8
gates verified up to date).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…gs are a closed contract — unknown keys throw (#4371 option 2)
All six engine methods now reject non-null option keys they do not execute,
naming the legal set per method; cursor/distinct quote their #4286 tombstone;
null stays a withdrawal. Legal sets = schema keys + documented extras
(searchFields — now DECLARED on EngineQueryOptionsSchema, it was read and
sent all along; onFieldsDropped on update; driver pass-through keys on the
methods whose bag reaches driver options — count/aggregate forward nothing,
so they reject those too). A drift pin holds the sets equal to the schemas.
Same sweep also closes what the option-2 caller survey turned up:
- bag object no longer overrides the resolved AST object ({...query, object}
order fix); protocol refuses a contradicting POST-body object (400
QUERY_OBJECT_MISMATCH) and strips its own vocabulary (object/count/
tombstones/non-aggregate having) off the engine bag
- nested expand ASTs reject the four wire-only spellings like the top level
- $search/$searchFields engine reads removed (protocol normalizes to bare)
- DbQueueAdapter purge/purgeFailed passed {id} — purge deleted NOTHING
(throw swallowed into warn), purgeFailed always threw; both now pass
{where: {id}}, and the test fake stops accepting the signature the real
engine rejects
- 19 test bags carrying the dead 'filters' key cleaned up (two were
intent-carrying: a locale multi-update and an oidc dogfood assert that
both silently scanned unfiltered)
- driver.find JSDoc example shows canonical QueryAST keys
Closes#4371 (option 1 landed in #4387).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vercel

vercelBot commented Jul 31, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredJul 31, 2026 11:19am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 4 package(s): @objectstack/metadata-protocol, @objectstack/objectql, @objectstack/service-queue, @objectstack/spec.

110 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx(via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx(via @objectstack/spec)
  • content/docs/ai/skills.mdx(via @objectstack/spec)
  • content/docs/api/client-sdk.mdx(via @objectstack/spec)
  • content/docs/api/environment-routing.mdx(via @objectstack/spec)
  • content/docs/api/error-catalog.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx(via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx(via @objectstack/spec)
  • content/docs/api/index.mdx(via @objectstack/spec)
  • content/docs/automation/approvals.mdx(via @objectstack/spec)
  • content/docs/automation/connectors.mdx(via @objectstack/spec)
  • content/docs/automation/flows.mdx(via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx(via packages/spec)
  • content/docs/automation/hooks.mdx(via @objectstack/spec)
  • content/docs/automation/index.mdx(via @objectstack/spec)
  • content/docs/automation/webhooks.mdx(via @objectstack/spec)
  • content/docs/automation/workflows.mdx(via @objectstack/spec)
  • content/docs/concepts/architecture.mdx(via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx(via packages/spec)
  • content/docs/concepts/index.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx(via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx(via @objectstack/metadata-protocol, @objectstack/objectql, packages/spec)
  • content/docs/concepts/north-star.mdx(via packages/spec)
  • content/docs/data-modeling/analytics.mdx(via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx(via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx(via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx(via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx(via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx(via packages/objectql, @objectstack/spec)
  • content/docs/data-modeling/index.mdx(via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx(via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx(via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx(via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx(via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx(via @objectstack/spec)
  • content/docs/deployment/cli.mdx(via @objectstack/spec)
  • content/docs/deployment/migration-from-objectql.mdx(via @objectstack/objectql)
  • content/docs/deployment/troubleshooting.mdx(via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx(via @objectstack/spec)
  • content/docs/deployment/vercel.mdx(via @objectstack/objectql)
  • content/docs/getting-started/build-with-claude-code.mdx(via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx(via @objectstack/spec)
  • content/docs/getting-started/examples.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx(via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx(via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx(via @objectstack/spec)
  • content/docs/kernel/cluster.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx(via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx(via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx(via packages/spec)
  • content/docs/kernel/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx(via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx(via packages/spec)
  • content/docs/kernel/services-checklist.mdx(via @objectstack/metadata-protocol, @objectstack/objectql, @objectstack/service-queue, @objectstack/spec)
  • content/docs/kernel/services.mdx(via @objectstack/objectql, @objectstack/spec)
  • content/docs/permissions/authentication.mdx(via @objectstack/objectql)
  • content/docs/permissions/authorization.mdx(via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx(via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx(via @objectstack/spec)
  • content/docs/permissions/positions.mdx(via @objectstack/spec)
  • content/docs/permissions/rls.mdx(via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx(via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx(via @objectstack/spec)
  • content/docs/plugins/development.mdx(via @objectstack/spec)
  • content/docs/plugins/index.mdx(via @objectstack/objectql, @objectstack/spec)
  • content/docs/plugins/packages.mdx(via @objectstack/objectql, @objectstack/service-queue, @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx(via @objectstack/spec)
  • content/docs/protocol/diagram.mdx(via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx(via @objectstack/objectql, @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx(via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx(via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx(via packages/objectql, @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx(via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx(via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx(via @objectstack/objectql, @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx(via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx(via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx(via @objectstack/objectql, @objectstack/service-queue, @objectstack/spec)
  • content/docs/releases/index.mdx(via @objectstack/spec)
  • content/docs/releases/v12.mdx(via @objectstack/spec)
  • content/docs/releases/v13.mdx(via @objectstack/spec)
  • content/docs/releases/v16.mdx(via @objectstack/spec)
  • content/docs/releases/v17.mdx(via @objectstack/spec)
  • content/docs/releases/v9.mdx(via @objectstack/metadata-protocol, @objectstack/spec)
  • content/docs/ui/actions.mdx(via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx(via @objectstack/spec)
  • content/docs/ui/dashboards.mdx(via @objectstack/spec)
  • content/docs/ui/forms.mdx(via @objectstack/spec)
  • content/docs/ui/index.mdx(via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx(via @objectstack/spec)
  • content/docs/ui/setup-app.mdx(via @objectstack/spec)
  • content/docs/ui/translations.mdx(via @objectstack/spec)
  • content/docs/ui/views.mdx(via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation protocol:data tests tooling size/l labels Jul 31, 2026
@os-zhuang
os-zhuang merged commit 58a03d2 into mainJul 31, 2026
18 checks passed
@os-zhuang
os-zhuang deleted the claude/engine-unknown-key-enforcement-b44637 branch July 31, 2026 11:31
os-zhuang added a commit that referenced this pull request Aug 1, 2026
…, and executes every option it declares (#4419) (#4459)
findOne reads a single row, which makes its predicate the only thing between
the caller and an arbitrary record. When the predicate is missing the result is
not `null` — it is the object's FIRST ROW: a real, plausible-looking record
with nothing to do with the request, which the `if (!row)` check every call
site already has cannot catch, and which then propagates into whatever is
computed next.
#4419 reported this against the `filter` key, which #4346 (fold on every entry
point) and #4400 (unknown keys throw) already closed. This is what those left
standing.
BREAKING: findOne refuses a query that selects nothing in particular.
findOne(o) / findOne(o, {}) / findOne(o, { where: {} })
-> findOne(o, { where: … }) the record matching this
-> findOne(o, { search: 'Acme' }) the record this search finds
-> findOne(o, { orderBy: [{ field, order }] }) the FIRST in this order
-> find(o, { limit: 1 }) any row will do, said aloud
The error names all four. `find` and `count` are unchanged — returning or
counting every row is an honest answer. The guard reads the CALLER's predicate,
before RLS/sharing middleware injects its own.
Two silent drops that produced the same wrong record are fixed with it:
- findOne({ search }) now applies the search. The ADR-0061 expansion lived in
find() alone while both methods are checked against the SAME legal-key set,
so `search` passed the gate, reached a driver that does not read it, and the
read ran unpredicated.
- MongoDBDriver.findOne now applies orderBy, fields and offset. It translated
`where` and dropped the rest, so "the newest record" returned whichever
document the scan reached first. No ordering is imposed when the caller
supplies none (#4363) — unchanged on both drivers.
And a drift pin walks ENGINE_OPTION_KEY_SETS.findOne requiring each declared
key to have an observable effect, so the next declared-but-unexecuted key fails
CI instead of shipping. The Mongo cases live in one shared table read by both a
server-free options suite and the real-mongod suite, so neither half can drift.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationprotocol:datasize/lteststooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] A direct engine call silently drops sort/select/skip/populate — declared query contract, zero enforcement

1 participant

@os-zhuang