Uh oh!
There was an error while loading. Please reload this page.
fix(service-analytics): apply the dataset-level filter on the ObjectQL analytics path (#10413 phase 1) - #10758
Conversation
…L path (#10413 phase 1) `ObjectQLStrategy.execute` built its engine filter from `normalizeAnalyticsFilterTree(query)` alone — the caller's `where` and the time windows — and consulted the dataset registry nowhere. On a deployment whose driver reports `objectqlAggregate` but not `nativeSql`, `engine.aggregate` was therefore called with no `filter` key at all: the dataset's definition-level scope was dropped and every measure aggregated the whole table, while the dashboard door answered the scoped numbers for the same cube. The scope now travels as its own `$and` conjunct — never a key merge, for the reason `withReadScope` states: the caller's `where` and the dataset scope can name the same field. The representative SQL echo renders it as well, on the #3601/#3602 rule that an echo omitting an applied predicate is the same lie as one inventing a predicate. Phase 1 only: per-measure filters cannot be expressed against an aggregation typed `{ field, method, alias }`. That contract widening is #10576 and the lowering is phase 2 — pinned open, wrong numbers and all, in `objectql-dataset-filter.test.ts`. Part of #10413 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PnJHU45vPJj5UQrxe946Bx
📓 Docs Drift CheckThis PR changes 1 package(s): ⛔ 2 release-owned page(s) name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 8 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin f022a72c25e2f416e85f00bcdcdf19d17b44c38e && git checkout f022a72c25e2f416e85f00bcdcdf19d17b44c38e
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 78ac958552d3749793dd7d4cc612c651cc7df2b9 5933b981cfd174370306ee262ae0092866c2fa4b && git checkout -B drift-repro 78ac958552d3749793dd7d4cc612c651cc7df2b9 && git merge --no-ff 5933b981cfd174370306ee262ae0092866c2fa4b
node scripts/docs-audit/affected-docs.mjs --json 78ac958552d3749793dd7d4cc612c651cc7df2b9
|
Uh oh!
There was an error while loading. Please reload this page.
Part of #10413 — phase 1 only. This card stays open for phase 2, which waits on #10576.
The three doors, measured in one tree and one run
A dataset carries two declarations the
Cubemodel has no room for: a definition-levelfilter(its intrinsic scope) and a per-measurefilter. Three doors read them, andbefore this change they disagreed three ways:
filterfilterNativeSQLStrategy)WHEREconjunctqueryDataset→DatasetExecutor)combineFiltersANDs it into the runtime filtersplitMeasuresByFilter/runMeasurePassfan out one query per filtered measureObjectQLStrategy)Reproduced on this branch's merge base (
53a48c93f) with a stubexecuteAggregatethatrecords its arguments:
svc.query({ cube: 'opportunity_metrics', measures: [...] })on aservice whose capabilities are
{ nativeSql: false, objectqlAggregate: true }reached theengine with
filter: undefined— no filter key at all. The pre-fix run of the new testfile failed with
expected undefined to deeply equal { '$and': [ { is_deleted: false } ] }.What this PR changes
ObjectQLStrategy.executenow reads the dataset scope from the same channelNativeSQLStrategyreads it from —getDatasetScope(cubeName)on the strategy context,added for #10298 — and ANDs the definition-level
filterinto the whole-call filterengine.aggregatealready accepts. No contract movement:engine.aggregatehas alwaystaken one predicate for the whole call.
Merge semantics, read from the code rather than assumed. The scope is pushed onto
conjunctsand folded by the line that already existed at the end of the filter build:so it is ANDed, never merged key-by-key — the posture
withReadScopestates in thesame file ("Composed with
$and, never by key merge: the query's own filter and the scopecan name the SAME field … a spread would let caller input silently overwrite"). The
dashboard door reaches the same semantics through
combineFiltersindataset-executor.ts(
if (a && b) return { $and: [a, b] }), so both doors AND. An empty scope isrepresented by absence, not by an empty object:
filterNodeToConditionreturnsnullfor anode that constrains nothing, and
withReadScopeonly marks a filter whenObject.keys(filter).length > 0— so a dataset with nofilterstill calls the engine withno
filterkey.The SQL echo renders it too.
generateSqlon this path already renders the read scopedeliberately, on the rule that "a rendering that contradicts execution is worse than no
rendering" (#3601 / #3602 / #3650) — leaving the newly-applied predicate out of the echo
would have been the same lie in the other direction. This is the one place the change goes
beyond the strictly minimal edit, and it is called out here for review: same defect class,
same file, same gate family, and the correct shape was already pinned by the read-scope
block eight lines below it.
Phase 2 is NOT done, and is pinned open rather than left silent
Per-measure filters still do not reach this door: an aggregation is
{ field, method, alias }and cannot carry a predicate. Folding one into the whole-call filter would narrow every
measure at once — trading a wrong
won_countfor a wrongopp_counttoo — someasureFiltersis deliberately unread here (the only occurrence of that identifier in thefile is the comment saying so; the sibling strategy that does read it has one code hit,
which is the positive control for that search).
objectql-dataset-filter.test.tstherefore contains a[#10413 phase 2 — NOT DONE]blockasserting the current, wrong numbers —
won_count: 24,won_amount: 5_632_500where thetruth is
8and1_290_000— with a comment naming #10576 and instructing that thoseassertions be flipped, not deleted, when the contract widens. Without it this path reads
all-green while still answering the whole book for won revenue.
Pins
engine.aggregatefilterequals{ $and: [{ is_deleted: false }] }filterisundefinedwhereand the time windows still merge in and are not clobbered{ stage: 'closed_won', $and: [{ is_deleted: false }] }, and 8 rows — not 11 (scope dropped) and not 24 (whereclobbered); adateRangestill lands on its own fieldis_deletedconjunct, bound once — not doubled{alias, field, method};closed_wonreaches the engine nowhereWHERE is_deleted = $1,params: [false]; nothing extra for an unscoped datasetVerification
pnpm --filter @objectstack/service-analytics exec vitest run— 79 files, 1748 tests, allpassing (14 of them new).
Ablation, predictions written before mutating, neither leg rebuilt because the subject is
source-resolved (the test imports
../analytics-service.jsrelatively andpackages/services/service-analytics/distdoes not exist — the mutation changing results withno build is itself the positive control):
conjuncts.push(scopeCondition)generateSqlBoth restored by
git checkoutand verified byte-identical withgit hash-object(
96b2bbb687a09f45ed6a7c59fdf930ec6f49bac2before mutation, after each restore, and at theend), with 14/14 green on the restored tree.
Gates —
node scripts/pm/dispatch-gates.mjswith no path arguments, on the final commit5933b981cwith a clean tree; exit codes captured before any pipe. All 17 exit 0:check:changeset-gate-self-tests,check:objectui-changeset,check:slot-lookup,check:test-source-alias,check:type-source-resolution,check-adr-0087-registration,check-changeset-no-major,check-empty-changeset,check-affected-docs,check:query-options-erasure,check:type-check-coverage,check:type-check-debt --re-measure,check:engine-double-contract,check:where-matcher,check:nul-bytes, pluscheck:route-envelope --self-testandcheck:dispatcher-error-vocabulary --self-test, which thederivation did not name (the known-short-union class #10309).
check:type-check-debt --re-measure: "33 ledger entr(ies) re-measured in 364.6s, 1912 raw tscerror(s) total, none above its recorded number." It reports
@objectstack/plugin-authaslowerable (records 109, measures 97) — untouched here on purpose, per #10615.
Generated by Claude Code