fix(spec,service-analytics): carry a percentage measure's SCALE, and report an empty filtered group as zero (objectui#3136)

.changeset/dataset-percent-scale-chain.md

@@ -0,0 +1,58 @@
+---
+"@objectstack/spec": minor
+"@objectstack/service-analytics": minor
+---
+
+fix(spec,service-analytics): a percentage measure carries its SCALE, so a ratio of 1 is 100% (objectui#3136)
+
+A `%` format string says how to PRINT a number, not what scale that number is
+on — and the two readings collide at exactly `1`, which is both "100%" (a 0–1
+ratio at full compliance) and "1%" (a single percentage point). With nothing on
+the wire to tell them apart, renderers guessed from the value's magnitude and
+resolved the collision the wrong way: an SLA / pass-rate dashboard reporting
+`sla_rate = 1` displayed **"1.0%"** — "everything met the SLA" read as "1% met
+the SLA" — on both the KPI card and the dataset table.
+
+The scale was never actually unknowable; it just never left the server. A
+measure declaring `derived: { op: 'ratio' }` is a 0–1 fraction *by definition*,
+and a measure aggregating a `percent` field has whatever scale that field
+stores. Both facts sit in metadata the enrichment pass already reads for the
+ADR-0053 currency chain — which walks back to the source field, checks
+`type === 'currency'`, and rides the resolved code onto the result column.
+Percentages got no such treatment. They do now, through the same seam.
+
+**`percentScaleOf(field)` (`@objectstack/spec/data`)** is the one place the
+question is answered. A `percent` field stores a FRACTION unless it declares
+`max > 1` (e.g. `min: 0, max: 100`), which marks whole-percent storage — the
+same rule the percent edit widget already writes by, so a value round-trips.
+Non-`percent` fields get no opinion: a plain `number` an author formatted with
+a `%` keeps meaning exactly what their format string says.
+
+**`AnalyticsResult.fields[].percentScale`** carries the answer: `'fraction'`
+(`1` ⇒ "100%") or `'whole'` (`1` ⇒ "1%"), absent when the column is not a
+percentage. `queryDataset` sets it from the measure's `derived.op === 'ratio'`
+first, then the source field's scale. `currency` — emitted since ADR-0053 but
+only ever written through a cast — is now declared on the same interface.
+
+The config seam `measureCurrency` is renamed **`sourceFieldMeta`** and returns
+`max` alongside `type`/`defaultCurrency`. The old name had already outgrown
+itself: the date-bucketing path reads `type` through it to tell a `date`
+dimension from a `datetime` one, and the percent chain is its third consumer.
+
+Renderers that receive `percentScale` must scale by it rather than inferring
+from the value; one that does not receive it (an older server) keeps whatever
+fallback it has, so this is additive on the wire.
+
+**Same widget family, second fix: an empty filtered group is a measured zero.**
+A measure-scoped filter can exclude every row of a group the grid still lists,
+and the database reports that by omitting the group from the supplementary
+result — after the merge, indistinguishable from "not measured". For a COUNT or
+a SUM it *is* measured: the answer is 0. `emptyGroupValueFor(aggregate)`
+(`spec/data/aggregation-policy`) states which aggregates have an identity over
+the empty set, and `queryDataset` fills it in once all supplementary merges are
+done (a later measure's merge can append rows no earlier query saw). So
+"0 of 12 paid" now reports `0` instead of blank, and a ratio built on it
+computes to `0` instead of going null — the difference between a dashboard
+saying "0% met the SLA" and saying nothing at all. `avg`/`min`/`max` keep their
+null: there is nothing to average over an empty group, and flattening that to
+zero would invent a measurement.

.changeset/spec-api-surface-baseline-percent-scale.md

@@ -0,0 +1,11 @@
+---
+---
+
+Regenerates `packages/spec/api-surface.json` so the committed baseline records the
+four `./data` exports the percent-scale chain adds — `PercentScale`,
+`PercentScaleFieldMeta`, `percentScaleOf` and `emptyGroupValueFor`.
+
+Deliberately empty: this releases nothing. `api-surface.json` is a build-time
+snapshot the `check:api-surface` gate diffs against, not shipped code, and the
+exports it now records are already described by the changeset for the change that
+introduced them. Declaring a bump here would double-count that release.

examples/app-showcase/src/system/translations/index.ts

@@ -160,6 +160,12 @@ export const ShowcaseTranslationBundle = {
  combo_count_vs_progress: { title: 'Task Count vs Avg Progress' },
  },
  },
+ showcase_revenue_pulse: {
+ widgets: {
+ kpi_paid_rate: { title: 'Paid Rate' },
+ table_rate_by_status: { title: 'Paid Rate by Status' },
+ },
+ },
  },
  },
  'zh-CN': {
@@ -376,6 +382,16 @@ export const ShowcaseTranslationBundle = {
  combo_count_vs_progress: { title: '任务数与平均进度' },
  },
  },
+ // Same rule as the gallery above: these two widgets are born with the
+ // percent-scale fix, so they are translated at birth. Revenue Pulse's
+ // other widget titles predate the ratchet and stay in the frozen
+ // baseline. objectui#3136.
+ showcase_revenue_pulse: {
+ widgets: {
+ kpi_paid_rate: { title: '已付比例' },
+ table_rate_by_status: { title: '各状态已付比例' },
+ },
+ },
  },
  },
 };

examples/app-showcase/src/ui/dashboards/revenue-pulse.dashboard.ts

@@ -64,5 +64,14 @@ export const RevenuePulseDashboard: Dashboard = {
  { id: 'col_accounts_by_month', type: 'column', title: 'Accounts Signed by Month', dataset: accountDs, dimensions: ['signed_on'], values: ['account_count'], chartConfig: cfg('column', 'signed_on', 'account_count'), filterBindings: { dateRange: 'signed_on', region: 'sales_region' }, layout: { x: 6, y: 2, w: 6, h: 4 } },
  { id: 'donut_invoices_by_status', type: 'donut', title: 'Invoices by Status', dataset: invoiceDs, dimensions: ['status'], values: ['invoice_count'], chartConfig: cfg('donut', 'status', 'invoice_count'), layout: { x: 0, y: 6, w: 6, h: 4 } },
  { id: 'bar_accounts_by_industry', type: 'bar', title: 'Accounts by Industry', dataset: accountDs, dimensions: ['industry'], values: ['account_count'], chartConfig: cfg('bar', 'industry', 'account_count'), filterBindings: { dateRange: 'signed_on', region: 'sales_region' }, layout: { x: 6, y: 6, w: 6, h: 4 } },
+
+ // ── Percent scale (objectui#3136) ────────────────────────────────────
+ // A ratio measure rendered on the two surfaces that disagreed: a KPI card
+ // and a grouped table. Grouping by status pins the Paid row's rate at
+ // exactly 1 — the boundary where "is this a 0–1 ratio or a percentage
+ // point?" cannot be answered from the number, and where guessing printed
+ // "1.0%". The column now carries its declared scale, so it reads 100.0%.
+ { id: 'kpi_paid_rate', type: 'metric', title: 'Paid Rate', dataset: invoiceDs, values: ['paid_rate'], colorVariant: 'orange', layout: { x: 0, y: 10, w: 3, h: 2 } },
+ { id: 'table_rate_by_status', type: 'table', title: 'Paid Rate by Status', dataset: invoiceDs, dimensions: ['status'], values: ['invoice_count', 'paid_count', 'paid_rate'], layout: { x: 3, y: 10, w: 9, h: 4 } },
  ],
 };

examples/app-showcase/src/ui/datasets/revenue-pulse.dataset.ts

@@ -24,6 +24,18 @@ export const ShowcaseInvoiceDataset = defineDataset({
  measures: [
  { name: 'invoice_count', label: 'Invoices', aggregate: 'count' },
  { name: 'subtotal_sum', label: 'Subtotal', aggregate: 'sum', field: 'total', format: '0,0' },
+ // A RATE — the percent-scale case (objectui#3136). `paid_rate` is a 0–1
+ // ratio by construction, and grouping by `status` makes the Paid bucket
+ // exactly 1: the value a magnitude-guessing renderer printed as "1.0%"
+ // instead of "100.0%". The server annotates the column's scale from the
+ // `ratio` operator, so display no longer has to infer it.
+ { name: 'paid_count', label: 'Paid Invoices', aggregate: 'count', filter: { status: 'paid' } },
+ {
+ name: 'paid_rate',
+ label: 'Paid Rate',
+ derived: { op: 'ratio', of: ['paid_count', 'invoice_count'] },
+ format: '0.0%',
+ },
  ],
 });
 

packages/services/service-analytics/src/__tests__/dataset-granularity-postprocess.test.ts

@@ -58,7 +58,7 @@ function service(bucketByGranularity: Record<string, string>) {
  return [{ created_at: bucketByGranularity[g ?? 'none'], task_count: 10, account_count: 10 }];
  },
  // `created_at` is a tz-naive calendar date → ranges are exact under any tz.
-measureCurrency: (_o, f) => (f === 'created_at' ? { type: 'date' } : undefined),
+sourceFieldMeta: (_o, f) => (f === 'created_at' ? { type: 'date' } : undefined),
  });
  return { svc, seen };
 }

packages/services/service-analytics/src/__tests__/query-dataset.test.ts

@@ -123,12 +123,12 @@ describe('AnalyticsService.queryDataset', () => {
  });
 
  // ── ADR-0053 currency chain (measure → field currencyConfig → tenant ctx) ──
- function pricedSvc(rows: Array<Record<string, unknown>>, measureCurrency?: (o: string, f: string) => { type?: string; defaultCurrency?: string } | undefined) {
+ function pricedSvc(rows: Array<Record<string, unknown>>, sourceFieldMeta?: (o: string, f: string) => { type?: string; defaultCurrency?: string; max?: number } | undefined) {
  return new AnalyticsService({
  queryCapabilities: () => ({ nativeSql: true, objectqlAggregate: false, inMemory: false }),
  executeRawSql: async () => rows,
  getReadScope: (_o, ctx?: ExecutionContext) => (ctx?.tenantId ? { organization_id: ctx.tenantId } : undefined),
- ...(measureCurrency ? { measureCurrency } : {}),
+ ...(sourceFieldMeta ? { sourceFieldMeta } : {}),
  });
  }
  const moneyDataset = (measure: Record<string, unknown>) => DatasetSchema.parse({
@@ -161,6 +161,95 @@ describe('AnalyticsService.queryDataset', () => {
  expect(r.fields.find((f: any) => f.name === 'revenue')?.currency).toBeUndefined();
  });
 
+ // ── percent scale chain (objectui#3136) ───────────────────────────────────
+ // A "%" format says how to PRINT a number, not what scale it is on, and the
+ // two readings collide at exactly 1 ("100%" vs "1%"). The scale is answerable
+ // from metadata, so it rides onto the result column next to `currency`.
+ const rateDataset = (measures: Array<Record<string, unknown>>) => DatasetSchema.parse({
+ name: 'sla', label: 'SLA', object: 'ticket', include: [],
+ dimensions: [{ name: 'status', field: 'status', type: 'string' }],
+ measures,
+ });
+
+ it('marks a derived RATIO as fraction-scaled — the 1.0 = 100% case', async () => {
+ // Two of two met: the ratio is exactly 1, the value that renders as "1.0%"
+ // when a renderer guesses the scale from the number's magnitude.
+ const svc = pricedSvc([{ status: 'met', met_count: 2, base_count: 2 }]);
+ const r = await svc.queryDataset(
+ rateDataset([
+ { name: 'base_count', aggregate: 'count', label: 'Applicable' },
+ { name: 'met_count', aggregate: 'count', field: 'met', label: 'Met' },
+ { name: 'sla_rate', label: 'SLA rate', derived: { op: 'ratio', of: ['met_count', 'base_count'] }, format: '0.0%' },
+ ]),
+ { dimensions: ['status'], measures: ['base_count', 'met_count', 'sla_rate'] },
+ { tenantId: 'o' } as ExecutionContext,
+ ) as any;
+ expect(r.rows[0].sla_rate).toBe(1);
+ expect(r.fields.find((f: any) => f.name === 'sla_rate')?.percentScale).toBe('fraction');
+ // A count is not a percentage — annotating it would be a lie about the scale.
+ expect(r.fields.find((f: any) => f.name === 'base_count')?.percentScale).toBeUndefined();
+ });
+
+ it('a measure-scoped COUNT over a group with no matching rows is 0, not blank', async () => {
+ // The supplementary query for `met_count` returns only the groups that had
+ // a matching row; the database reports "none matched" by omitting the group
+ // entirely. That omission is a measured ZERO for a count — reporting it as
+ // missing blanked the cell and left the ratio null, hiding "0% of breached
+ // tickets met the SLA" on the one dashboard that exists to show it.
+ const svc = new AnalyticsService({
+ queryCapabilities: () => ({ nativeSql: true, objectqlAggregate: false, inMemory: false }),
+ // The base pass sees both groups; the filtered pass only "met".
+ executeRawSql: async (_o, sql) => sql.includes('met_count')
+ ? [{ status: 'met', met_count: 2 }]
+ : [{ status: 'met', base_count: 2 }, { status: 'breached', base_count: 3 }],
+ getReadScope: () => undefined,
+ });
+ const r = await svc.queryDataset(
+ rateDataset([
+ { name: 'base_count', aggregate: 'count', label: 'Applicable' },
+ { name: 'met_count', aggregate: 'count', field: 'met', label: 'Met', filter: { sla_met: true } },
+ { name: 'sla_rate', label: 'SLA rate', derived: { op: 'ratio', of: ['met_count', 'base_count'] }, format: '0.0%' },
+ ]),
+ { dimensions: ['status'], measures: ['base_count', 'met_count', 'sla_rate'] },
+ { tenantId: 'o' } as ExecutionContext,
+ ) as any;
+ const breached = r.rows.find((x: any) => x.status === 'breached');
+ expect(breached.met_count).toBe(0);
+ expect(breached.sla_rate).toBe(0);
+ // The group that DID match is untouched — and is the 1.0 = 100% case.
+ expect(r.rows.find((x: any) => x.status === 'met').sla_rate).toBe(1);
+ });
+
+ it('inherits the SOURCE FIELD scale: a `max: 100` percent field is whole-scaled', async () => {
+ const svc = pricedSvc([{ status: 'open', allocation: 80 }], (_o, f) => f === 'allocation_percent' ? { type: 'percent', max: 100 } : undefined);
+ const r = await svc.queryDataset(
+ rateDataset([{ name: 'allocation', aggregate: 'avg', field: 'allocation_percent', label: 'Allocation', format: '0.0%' }]),
+ { dimensions: ['status'], measures: ['allocation'] },
+ { tenantId: 'o' } as ExecutionContext,
+ ) as any;
+ expect(r.fields.find((f: any) => f.name === 'allocation')?.percentScale).toBe('whole');
+ });
+
+ it('inherits the SOURCE FIELD scale: a bare percent field is fraction-scaled', async () => {
+ const svc = pricedSvc([{ status: 'open', win: 0.75 }], (_o, f) => f === 'win_probability' ? { type: 'percent' } : undefined);
+ const r = await svc.queryDataset(
+ rateDataset([{ name: 'win', aggregate: 'avg', field: 'win_probability', label: 'Win', format: '0.0%' }]),
+ { dimensions: ['status'], measures: ['win'] },
+ { tenantId: 'o' } as ExecutionContext,
+ ) as any;
+ expect(r.fields.find((f: any) => f.name === 'win')?.percentScale).toBe('fraction');
+ });
+
+ it('leaves a plain-number measure unannotated — its format string stays the only word', async () => {
+ const svc = pricedSvc([{ status: 'open', tax: 7 }], (_o, f) => f === 'tax_rate' ? { type: 'number', max: 100 } : undefined);
+ const r = await svc.queryDataset(
+ rateDataset([{ name: 'tax', aggregate: 'avg', field: 'tax_rate', label: 'Tax', format: '0.0%' }]),
+ { dimensions: ['status'], measures: ['tax'] },
+ { tenantId: 'o' } as ExecutionContext,
+ ) as any;
+ expect(r.fields.find((f: any) => f.name === 'tax')?.percentScale).toBeUndefined();
+ });
+
  it('enriches dimension columns with their dataset display label', async () => {
  const labeled = DatasetSchema.parse({
  name: 'sales2', label: 'Sales', object: 'opportunity', include: ['account'],
@@ -259,7 +348,7 @@ describe('AnalyticsService.queryDataset', () => {
  // closed_at is a datetime instant → its month bucket boundary is that tz's
  // MIDNIGHT INSTANT. June/July 2026 in New York are EDT (−04), so local
  // midnight is 04:00 UTC.
-measureCurrency: (_o, f) => (f === 'closed_at' ? { type: 'datetime' } : undefined),
+sourceFieldMeta: (_o, f) => (f === 'closed_at' ? { type: 'datetime' } : undefined),
  getReadScope: (_o, ctx?: ExecutionContext) => (ctx?.tenantId ? { organization_id: ctx.tenantId } : undefined),
  });
  const result = await svc.queryDataset(
@@ -282,7 +371,7 @@ describe('AnalyticsService.queryDataset', () => {
  const svc = new AnalyticsService({
  queryCapabilities: () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false }),
  executeAggregate: async () => [{ close_date: '2026-06', revenue: 100 }],
-measureCurrency: (_o, f) => (f === 'close_date' ? { type: 'date' } : undefined),
+sourceFieldMeta: (_o, f) => (f === 'close_date' ? { type: 'date' } : undefined),
  getReadScope: (_o, ctx?: ExecutionContext) => (ctx?.tenantId ? { organization_id: ctx.tenantId } : undefined),
  });
  const result = await svc.queryDataset(