Skip to content

fix(service-analytics): 带 measure-scoped filter / derived 度量的 dataset 查询,fields 也描述维度列 (#5537) - #5691

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5537-dataset-dimension-fields
Aug 6, 2026
Merged

fix(service-analytics): 带 measure-scoped filter / derived 度量的 dataset 查询,fields 也描述维度列 (#5537)#5691
os-zhuang merged 1 commit into
mainfrom
claude/issue-5537-dataset-dimension-fields

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes#5537

前提重验(对 origin/main c36abfe,含 #5587 / #5634 / #5667

核心前提成立,但立单时的落点分析与影响面各有一处需要修正。

成立的部分。 A/B/C 三形态逐条复现,新测试在未改动源码时的结果就是立单描述:

A {dimensions:['owner'], measures:['opp_count']} → fields ['owner','opp_count'] 绿
B {dimensions:['owner'], measures:['won_count']} → fields ['won_count'] 红
C {dimensions:['lead_source'], measures:['won_count','decided_count','win_rate']}
→ fields ['won_count',…] 无维度 红

修正一 —— 触发条件比「带 filter 或 derived」更窄,而且 derived 不是独立成因。
真正的判据是「基础度量是否全部带自身 filter」。runMeasurePass 只在存在无 filter
度量时才发主查询;混合选择(['opp_count','won_count'])照旧发主查询,维度描述符一直都在
—— 该形态在修复前就是绿的,已作为守卫钉住。derived 度量本身也从不丢维度:它只在
依赖全部带 filter 时丢(C 的 win_rate),而一个依赖无 filter 度量的 ratio
(avg_deal)修复前后皆绿,也已钉住。所以立单里「一旦查询走了 measure-scoped filter
合并含 derived 度量的路径」这个 or 偏宽了;去找一条 derived 专属的缺陷路径会白跑。

修正二 —— 维度没有 format 可丢。 立单要求断言 {name,type,label,format},并把
「数值维度额外丢失 format」记为症状。DatasetDimensionSchema(packages/spec/src/ui/dataset.zod.ts)
只声明 name / label / field / type / dateGranularity —— 没有 format;
formatDatasetMeasureSchema 的键。AnalyticsResult.fields[].format 因此只由度量
富化产出,在任何路径上(含一直正常的 A)都不为维度产出。也就是说不存在「A 有、B 没有」
的维度 format,断言它等于断言一个幻影。日期维度的分桶显示是 resolveDimensionLabels
在服务端改写行值完成的,不走 field format。故本 PR 的维度描述符 = {name, type, label},
type 明确按例断言(日期轴是 time,不是渲染端默认的 string)—— 这是 A 路径实际产出的
形状,也就是收敛目标。

修正三 —— 立单指的两个可选落点里,「service 侧新建 field 条目」是错的方向。 见下。

修复位置的代码证据

产出 fields 的一共三处,且都是「维度在前,度量在后」:两个 strategy 的 buildFieldMeta
(objectql-strategy.ts L1138 / native-sql-strategy.ts L748)与草稿预览的
preview-evaluator.ts L250。投影集合的定义在 projectedDimensions(objectql-strategy.ts
L1130):每个 dimensions 条目,加上每个带 granularity 且不在其中的 timeDimensions 条目
—— #4033 的「one definition, every consumer」。

DatasetExecutor.runMeasurePass 的补充子查询已经拿到了这些描述符(sub.fields),
只是把它们扔了,只留 sub.rows 去 merge。因此选 executor 侧、并且从子结果里取:

if (!primary && i === 0) {
for (const f of sub.fields ?? []) {
if (!measureNames.has(f.name)) result.fields.push(f);
}
}

为什么不在 executor 里从 compiled.cube 重算:那会成为「哪些维度被投影」这条规则的第四份
拷贝
,可以与它所标注的那些行漂移(#4033 正是这个漂移的原始伤口)。读回子结果则让两条路径
按构造收敛 —— 顺序、type、以及当前 strategy 究竟投影了什么,全都一致,包括 native
与 objectql 两条 strategy 各自的答案。

为什么不在 service 侧「维度缺失时新建 field 条目」:那一段(analytics-service.ts L915 起)
只应富化,不应造列 —— 某列是否存在是查询层的答案,在这里凭 selection.dimensions
造一条,可能描述出行里根本没有的列(反之亦然:上面 #5688 那种只在 timeDimensions 里的列,
service 侧的 selectedDims 看不见,却确实是结果列)。已在该处补注释写明这条边界与 #5537
的因果。

字段序:两条路径现在都是 [维度…, 无 filter 度量…, 带 filter 度量…, __compare…, derived…],
按 exact toEqual 钉住(不是 toContain)。

反向验证(先预测方向,再跑)

预测:还原 executor 那一处 seam → 「基础度量全部带 filter」的 8 例转红,4 例收敛目标保持绿;
方向普通,无反转、无计数移动(本改动新增原本缺失的 field 条目,不收紧任何规则、不删除
任何 ?? 分支,故下游不可能因此出一个 finding)。

实测(git stash 仅还原源码,测试文件不动):8 failed | 4 passed (12) —— 与预测逐例吻合,
红的正是 B/C 块全部,绿的正是「已正常」块全部。

范围外发现(已立单,未搭车)

#5688 —— 一个只带 dateRangetimeDimensions 条目会被 buildQuery 补上 dataset
的默认粒度,于是「窗口」变成第二层 GROUP BY:{dimensions:['owner']} + 一个日期区间筛选
回来的是按 owner × 月拆开的行,还多一个没人选过的时间列。它在本 PR 从未触及的那条
A 路径上同样复现(已在 origin/main 与本分支上各测一次,输出逐字节相同),修它会改变响应
形状(行数,不是 label),不该作为本单的搭车项。本 PR 把它当数据钉住(含行数断言)并成对
放在两个 block 里,#5688 落地那天这对用例会有意转红。


Generated by Claude Code

…the measure-filter merge path (#5537)
A dataset selection whose base measures ALL carry their own measure-scoped
`filter` (or whose selected derived measures depend only on such measures)
never issues the primary grouped query: `runMeasurePass` starts from a
synthesized `{ rows: [], fields: [] }` and each supplementary sub-query appends
exactly one MEASURE descriptor. The dimension columns are in every row — they
are the merge key — but described nowhere, so a consumer reading column
metadata had no `label` and no `type` for the grouped column and fell back to
humanizing the raw key: "owner" where the dataset declares "Owner".
The dimension descriptors are now adopted from the FIRST supplementary result,
which projects exactly the dimensions the pass groups by. Taken from the
sub-result rather than re-derived from `compiled.cube`: the projected set is
defined by each strategy's `buildFieldMeta` (#4033), so an executor-side
reconstruction would be a fourth copy of that rule, free to drift from the rows
it labels. Reading it back makes the two paths converge by construction — on
order (dimensions first), on `type`, and on whatever the active strategy
projected. `compareTo`, `totals` and derived measures all route through the
same pass, so all three are covered.
The label enrichment in `queryDataset` is left enriching-only, with a note
saying why: whether a column exists is the query layer's answer, and minting
one there would describe a column the rows may not carry.
Tests: `dataset-dimension-field-descriptors.test.ts` asserts the complete
`fields` set and order (exact `toEqual`, full descriptor) for the healthy
single-query path and for every all-filtered shape. Reverse verification —
direction predicted first — reverting the executor seam turns the 8
all-filtered cases red and leaves the 4 convergence-target cases green;
measured exactly that.
Out of scope, filed as #5688: a `timeDimensions` entry carrying only a
`dateRange` acquires the dataset's default granularity, turning a WINDOW into a
second GROUP BY. Reproduces identically on the path this fix never touched;
pinned here as data (row count included) so #5688 landing goes red on purpose.
@vercel

vercelBot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
ProjectDeploymentActionsUpdated (UTC)
objectstackIgnoredIgnoredAug 6, 2026 12:23am

Request Review

@github-actionsgithub-actionsBot added documentation Improvements or additions to documentation tests tooling size/l labels Aug 6, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/service-analytics.

8 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/api/data-api.mdx(via @objectstack/service-analytics)
  • content/docs/api/index.mdx(via @objectstack/service-analytics)
  • content/docs/kernel/services-checklist.mdx(via @objectstack/service-analytics)
  • content/docs/permissions/sharing-rules.mdx(via @objectstack/service-analytics)
  • content/docs/plugins/packages.mdx(via @objectstack/service-analytics)
  • content/docs/releases/implementation-status.mdx(via @objectstack/service-analytics)
  • content/docs/releases/v17.mdx(via @objectstack/service-analytics)
  • content/docs/releases/v9.mdx(via @objectstack/service-analytics)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuangClaude

Copy link
Copy Markdown
ContributorAuthor

真机线上验证(matched before/after,同一 dataset、同一请求体)

showcase 自带一个可复现的全 filter 形态:showcase_invoice_metrics
paid_count(aggregate: count, filter: { status: 'paid' })—— 只选它就是「基础度量
全部带 filter」。用 pnpm dev -- --fresh -p <随机端口> 起真服务,打 POST /api/v1/analytics/dataset/query。为了让 before/after 真的成对,pre-fix 那一轮是把
dataset-executor.ts切回 origin/main、只重建 @objectstack/service-analytics
再重启服务后打的(并核对过 dist 里确实不含本次改动)。

修复前

A) {"dimensions":["region"], "measures":["invoice_count"]} 无 filter,对照组
fields [{"name":"region","type":"string","label":"Region"},{"name":"invoice_count",…}]
rowkeys ["region","invoice_count"]
B) {"dimensions":["region"], "measures":["paid_count"]}
fields [{"name":"paid_count","type":"number","label":"Paid Invoices"}] <-- 维度没了
rowkeys ["region","paid_count"] <-- 行里一直有
D) {"dimensions":["issued_on"],"measures":["paid_count"]} 日期轴
fields [{"name":"paid_count","type":"number","label":"Paid Invoices"}] <-- 维度没了
rowkeys ["issued_on","paid_count"]

修复后

A) fields [{"name":"region","type":"string","label":"Region"},{"name":"invoice_count",…}] 不变
B) fields [{"name":"region","type":"string","label":"Region"},{"name":"paid_count","type":"number","label":"Paid Invoices"}]
D) fields [{"name":"issued_on","type":"time","label":"Issued"},{"name":"paid_count","type":"number","label":"Paid Invoices"}]

三例的 fields 名字集合现在都与 rowkeys 完全一致;D 还证实日期轴拿回的是 type: "time"
(不是渲染端默认的 string)与 label: "Issued"。对照组 A 逐字节未变,说明这不是把两条路径
都改了、而是把出错的那条收敛到正常那条。

四个临时服务(39517/39631/39742/39858)均已按端口持有者 PID 关停,已确认端口全部释放。


Generated by Claude Code

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentationImprovements or additions to documentationsize/lteststooling

Projects

None yet

2 participants

@os-zhuang@claude