You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The FILTER axis has no unmaterializable verdict: a where on a virtual formula field returns 0 rows silently, while sort and search refuse the same field with a 400 #8296
Found while implementing #7226 (PR #8295), where this gap is what made that card's defect invisible. Filed unassigned; the fix is a platform change in packages/metadata-protocol, out of scope for an examples-only PR.
The asymmetry
formula is the one field type no driver materialises a column for. Three query axes can name a field; two of them refuse a formula loudly, one accepts it and answers nothing:
A formula field IS known, so it passes the door and reaches the driver, which has no column for it.
Measured
Real ObjectQL over @objectstack/driver-sqlite-wasm, one object, two rows — is_completed a formula whose source is record.status == "completed", status a stored text column:
where { is_completed: true } -> 0 rows, no error
where { is_completed: false } -> 0 rows, no error
CONTROL where { status: 'completed' } -> 1 row
CONTROL where { status: { $ne: 'completed' } } -> 1 row
The formula READS correctly on the same rows in the same call (applyFormulaPlan hydrates it), which is what makes this hard to spot: the field is visibly populated in the response and simultaneously unfilterable.
packages/spec/src/data/search-fields.ts already records the same measurement for the $contains case ("0 rows on driver-memory ... 0 rows with NO error on driver-sql/better-sqlite3"), so the storage fact is established repo-side; only the filter axis never grew a door for it.
Why it matters
Both directions are wrong, and the false direction is the dangerous one:
{ formula_field: true } returns nothing — a surface that looks merely empty.
{ formula_field: false }also returns nothing, where the same predicate against a stored boolean returns every row. A filter meaning "not yet done" silently becomes "no records at all".
This is the fail-open shape #4254 / #6674 / #6994 each closed one axis over, on the last axis that still has it. It is also a live authoring trap: #7226's app-todo had eight filters on that exact predicate, and the natural repair for that card — deriving the flags as formula fields — would have silently emptied a list view, a scheduled reminder flow and two reports, with no error anywhere. The fix there had to route around the platform rather than use it.
Shape of the fix, if wanted
Give assertFilterFieldsExist the third verdict its two neighbours already have, splitting unknown from virtual, judged by the same @objectstack/spec/data predicate the search axis uses (isVirtualSearchField / SEARCH_VIRTUAL_TYPES) so gate and drivers cannot disagree about which types have a column. The remedy sentence should match the wording the sort and search refusals already share ("denormalise onto a stored field ... and filter that"), per the one-vocabulary-across-doors discipline recorded on assertSortFieldsExist.
Blast radius. Unlike sort, a filter changes the row SET, so a refusal turns today's silent-zero surfaces into loud 4xx. That is the correct direction (ADR-0032, no silent failure), but it will surface every existing app that has one of these, which is a migration question rather than a pure bug fix.
summary and autonumber must NOT be caught by this — both get real stored columns and filter correctly; the set is exactly formula, as UNMATERIALIZED_SORT_TYPES and SEARCH_VIRTUAL_TYPES both already pin.
Found while implementing #7226 (PR #8295), where this gap is what made that card's defect invisible. Filed unassigned; the fix is a platform change in
packages/metadata-protocol, out of scope for an examples-only PR.The asymmetry
formulais the one field type no driver materialises a column for. Three query axes can name a field; two of them refuse a formula loudly, one accepts it and answers nothing:formulafieldassertSortFieldsExist+assertOrderByIsMaterializable400 INVALID_SORT, with a denormalize-onto-a-stored-field remedy (#6994, #7095)assertSearchFieldsAreSearchableassertFilterFieldsExistassertFilterFieldsExist(packages/metadata-protocol/src/protocol.ts) computes exactly one verdict:A
formulafield IS known, so it passes the door and reaches the driver, which has no column for it.Measured
Real
ObjectQLover@objectstack/driver-sqlite-wasm, one object, two rows —is_completedaformulawhose source isrecord.status == "completed",statusa stored text column:The formula READS correctly on the same rows in the same call (
applyFormulaPlanhydrates it), which is what makes this hard to spot: the field is visibly populated in the response and simultaneously unfilterable.packages/spec/src/data/search-fields.tsalready records the same measurement for the$containscase ("0 rows on driver-memory ... 0 rows with NO error on driver-sql/better-sqlite3"), so the storage fact is established repo-side; only the filter axis never grew a door for it.Why it matters
Both directions are wrong, and the
falsedirection is the dangerous one:{ formula_field: true }returns nothing — a surface that looks merely empty.{ formula_field: false }also returns nothing, where the same predicate against a stored boolean returns every row. A filter meaning "not yet done" silently becomes "no records at all".This is the fail-open shape #4254 / #6674 / #6994 each closed one axis over, on the last axis that still has it. It is also a live authoring trap: #7226's
app-todohad eight filters on that exact predicate, and the natural repair for that card — deriving the flags as formula fields — would have silently emptied a list view, a scheduled reminder flow and two reports, with no error anywhere. The fix there had to route around the platform rather than use it.Shape of the fix, if wanted
Give
assertFilterFieldsExistthe third verdict its two neighbours already have, splittingunknownfromvirtual, judged by the same@objectstack/spec/datapredicate the search axis uses (isVirtualSearchField/SEARCH_VIRTUAL_TYPES) so gate and drivers cannot disagree about which types have a column. The remedy sentence should match the wording the sort and search refusals already share ("denormalise onto a stored field ... and filter that"), per the one-vocabulary-across-doors discipline recorded onassertSortFieldsExist.Two things worth deciding rather than assuming:
engine.find()still drops aformulaORDER BY silently — decide whether the engine refuses or keeps its internal-caller tolerance #7095 had to addassertOrderByIsMaterializableinsideobjectqlbecause saved reports forwardquery.orderBystraight intoengine.find, bypassing the REST ingress. Filters are forwarded the same way from reports, flows and dashboards, so an ingress-only fix would leave the author-reachable half open — the same lesson, one axis over.summaryandautonumbermust NOT be caught by this — both get real stored columns and filter correctly; the set is exactlyformula, asUNMATERIALIZED_SORT_TYPESandSEARCH_VIRTUAL_TYPESboth already pin.Generated by Claude Code