Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .changeset/analytics-query-bare-shape-entry-validation.md
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
---
"@objectstack/spec": major
"@objectstack/runtime": minor
---

fix(spec,runtime)!: `AnalyticsQueryRequest` is the bare `AnalyticsQuery`; the dispatcher validates `/analytics` bodies at the entry (#3878)

**Spec.** `AnalyticsQueryRequestSchema` used to describe a
`{ cube, query: {...}, format }` ENVELOPE — the dialect of the retired degraded
analytics shim (#3891), which the real engine never understood: an envelope
body inferred a column-less cube and died as an SQL syntax error
(`SELECT FROM …`) instead of a shape error. The schema now describes what the
engine and every real caller actually use — the **bare `AnalyticsQuery`**:

```
FROM { "cube": "orders", "query": { "measures": ["count"] }, "format": "json" }
TO { "cube": "orders", "measures": ["count"], "dimensions": [...], "where": {...} }
```

`cube` + `measures` are required at the top level; `dimensions` / `where` /
`timeDimensions` / `order` / `limit` / `offset` / `timezone` sit beside them.
The schema is `.strict()`; `query` and `format` are tombstoned (`retiredKey`)
so both `tsc` and the parse answer with this exact migration. `format` was
never implemented (every response is the JSON envelope) — for CSV/XLSX use the
export surface. The removal is registered as two step-17 semantic migrations
(`analytics-query-request-envelope-retired`,
`analytics-query-request-format-retired`) — it is an HTTP-wire change with no
stored metadata to rewrite.

**Runtime.** `POST /api/v1/analytics/query` and `/analytics/sql` now validate
the body against that schema AT THE ENTRY and answer
**400 `VALIDATION_FAILED`** with per-field details — including the envelope
prescription above, and a bespoke hint that `filters` is not a contract field
(the filter field is `where`, the same canonical FilterCondition `find()`
takes). Previously a malformed body reached the engine and failed as a 500 SQL
syntax error, or had its off-contract filter key silently ignored. A valid
body is forwarded to the analytics service byte-identical (validation only —
parsing would inject the schema's `timezone: 'UTC'` default and override
org-timezone resolution). An uninstalled analytics capability still answers
404 before any body inspection (#3891).
12 changes: 10 additions & 2 deletions content/docs/references/api/analytics.mdx
Original file line numberDiff line numberDiff line change
Expand Up@@ -58,9 +58,17 @@ const result = AnalyticsEndpoint.parse(data);

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **query** | `{ cube?: string; measures: string[]; dimensions?: string[]; where?: any; … }` | ✅ | The analytic query definition |
| **cube** | `string` | ✅ | Target cube name |
| **format** | `Enum<'json' \| 'csv' \| 'xlsx'>` | ✅ | Response format |
| **measures** | `string[]` | ✅ | List of metrics to calculate |
| **dimensions** | `string[]` | optional | List of dimensions to group by |
| **where** | `any` | optional | Filtering criteria (canonical Query DSL FilterCondition) |
| **timeDimensions** | `{ dimension: string; granularity?: Enum<'second' \| 'minute' \| 'hour' \| 'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>; dateRange?: string \| string[] }[]` | optional | |
| **order** | `Record<string, Enum<'asc' \| 'desc'>>` | optional | |
| **limit** | `number` | optional | |
| **offset** | `number` | optional | |
| **timezone** | `string` | ✅ | |
| **query** | `any` | optional | [REMOVED] `query` was removed from AnalyticsQueryRequest in @objectstack/spec 17.0.0 (#3878). The `{ cube, query: {...}` } envelope was the dialect of the retired degraded analytics shim (#3891) — the real engine never understood it. Move the query.* fields to the body top level: `{ cube, measures, dimensions?, where?, timeDimensions?, order?, limit?, offset?, timezone? }`. |
| **format** | `any` | optional | [REMOVED] `format` was removed from AnalyticsQueryRequest in @objectstack/spec 17.0.0 (#3878). It was never implemented — every response is the JSON envelope. Delete the key; for CSV/XLSX use the export surface instead. |


---
Expand Down
11 changes: 11 additions & 0 deletions docs/protocol-upgrade-guide.md
Original file line numberDiff line numberDiff line change
Expand Up@@ -126,6 +126,8 @@ Beyond those spec-surface removals, it graduates the seven flow-node config key

And it removes the RLS-policy key `priority` (#3896 security audit): promised "conflict resolution" that cannot exist, because applicable policies OR-combine (most permissive wins) — there is never a conflict to order, and nothing ever read the key (call graph closed across the collection site, the projection round-trip and the compiler). A pure lossless delete: outcomes are identical with or without it; the schema tombstones the key with the same prescription.

On the wire contract it also retires the `/analytics/query` request ENVELOPE (#3878): `AnalyticsQueryRequestSchema` used to describe `{ cube, query: {...}, format }` — the dialect of the retired degraded analytics shim (#3891) that the real engine never understood (an envelope body inferred a column-less cube and died as an SQL syntax error). The canonical request body is now the BARE AnalyticsQuery — `cube` + `measures` at the top level — which is what every real caller already sends; the schema tombstones `query`/`format`, and the dispatcher entry validates bodies and answers 400 with the prescription. No stored metadata carries this shape (it was HTTP-only), so the change is two semantic TODOs for API callers rather than a stack conversion.

### Mechanical (applied for you)

| Conversion | Surface | Change | Load window |
Expand All@@ -140,6 +142,15 @@ And it removes the RLS-policy key `priority` (#3896 security audit): promised "c
| `flow-node-script-config-aliases` | `flow.node.script.config` | script flow-node config keys 'functionName' → 'function', 'input' → 'inputs' (#3796) | live — protocol 17 loader accepts the old shape |
| `permission-rls-priority-removed` | `permission.rowLevelSecurity.priority` | RLS-policy key 'priority' removed (#3896 audit — policies OR-combine, so the promised conflict-resolution semantics cannot exist; dropping it changes no outcome) | retired — `migrate meta` only |

### Semantic (delegated to you, with acceptance criteria)

- **`analytics-query-request-envelope-retired`** — `api.analyticsQueryRequest.query` → bare AnalyticsQuery body (top-level cube/measures/dimensions/where/...)
- Why not automatic: The { cube, query: {...} } envelope was an HTTP-wire dialect of the retired degraded analytics shim (#3891), never stored in stack metadata — there is no source for the chain to rewrite. Callers of POST /analytics/query and /analytics/sql must move the query.* fields to the body top level themselves.
- Done when: Every /analytics/query and /analytics/sql call sends the bare AnalyticsQuery shape and succeeds; no request answers 400 VALIDATION_FAILED with the envelope prescription.
- **`analytics-query-request-format-retired`** — `api.analyticsQueryRequest.format` → (removed — responses are always the JSON envelope; use the export surface for CSV/XLSX)
- Why not automatic: The `format` key was declared but never implemented (declared ≠ enforced): every response is the JSON envelope regardless of the requested value, so there is no behaviour to preserve and nothing stored to rewrite.
- Done when: No /analytics/query or /analytics/sql call sends `format`; exports go through the export surface.

---

*Machine-readable equivalents: `spec-changes.json` (shipped in `@objectstack/spec` and attached to each GitHub Release) and the structured output of `objectstack migrate meta --json`.*
Original file line numberDiff line numberDiff line change
Expand Up@@ -84,7 +84,9 @@ async function postAnalyticsQuery(err: unknown) {
expect(handler, 'POST /api/v1/analytics/query must be mounted').toBeTypeOf('function');

const res = makeRes();
await handler({ body: { cube: 'x', query: {} }, query: {} }, res);
// [#3878] Body must pass entry validation so the SERVICE's thrown error —
// the thing under test — is what reaches the exit, not an entry 400.
await handler({ body: { cube: 'x', measures: ['count'] }, query: {} }, res);
return res;
}

Expand Down
61 changes: 59 additions & 2 deletions packages/runtime/src/dispatcher-validation-error.real.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -20,7 +20,7 @@
* are the tests that fail if the contract moves.
*/

import { describe, it, expect } from 'vitest';
import { describe, it, expect, vi } from 'vitest';
import { ValidationError } from '@objectstack/objectql';

import { HttpDispatcher } from './http-dispatcher.js';
Expand DownExpand Up@@ -100,7 +100,9 @@ async function analyticsQuery(thrown: unknown) {
header() { return res; },
json(b: any) { res.body = b; return res; },
};
await handlers['POST /api/v1/analytics/query']({ body: { cube: 'x', query: {} }, query: {} }, res);
// [#3878] Body must pass entry validation so the SERVICE's thrown error —
// the thing under test — is what reaches the exit, not an entry 400.
await handlers['POST /api/v1/analytics/query']({ body: { cube: 'x', measures: ['count'] }, query: {} }, res);
return res;
}

Expand DownExpand Up@@ -138,3 +140,58 @@ describe('#3918 — both exits serve the real ValidationError as 400 + fields[]'
expect(res.body.error.details.fields).toEqual([]);
});
});

// ---------------------------------------------------------------------------

/** [#3878] Post `body` through the real route handler; the service never throws. */
async function postAnalyticsBody(body: unknown) {
const handlers: Record<string, (req: any, res: any) => any> = {};
const rec = (verb: string) => (path: string, h: any) => { handlers[`${verb} ${path}`] = h; };
const server = {
get: rec('GET'), post: rec('POST'), put: rec('PUT'),
delete: rec('DELETE'), patch: rec('PATCH'),
};
const query = vi.fn(async () => ({ rows: [] }));
const analytics = { query, getMeta: async () => ({ cubes: [] }), generateSql: async () => ({ sql: null }) };
const kernel = {
getService: (n: string) => (n === 'analytics' ? analytics : undefined),
getServiceAsync: async (n: string) => (n === 'analytics' ? analytics : undefined),
};
const plugin = createDispatcherPlugin({ prefix: '/api/v1', securityHeaders: false });
await plugin.start?.({
getKernel: () => kernel,
getService: (n: string) => (n === 'http.server' ? server : undefined),
environmentId: undefined,
logger: { info() {}, warn() {}, error() {}, debug() {} },
hook: () => {}, on: () => {},
} as any);

const res: any = {
statusCode: undefined, body: undefined,
status(c: number) { res.statusCode = c; return res; },
header() { return res; },
json(b: any) { res.body = b; return res; },
};
await handlers['POST /api/v1/analytics/query']({ body, query: {} }, res);
return { res, query };
}

describe('#3878 — entry validation reaches the wire as a 400, service untouched', () => {
it('the retired envelope answers 400 with the tombstone prescription', async () => {
const { res, query } = await postAnalyticsBody({ cube: 'x', query: { measures: ['count'] } });

expect(res.statusCode).toBe(400);
expect(res.body.error.code).toBe('VALIDATION_FAILED');
expect(res.body.error.message).toContain('top level');
expect(query).not.toHaveBeenCalled();
// A caller mistake, not a server fault: the reporter side-channel stays clear.
expect(res.__obsRecordedError).toBeUndefined();
});

it('a valid bare body passes through and answers 200', async () => {
const { res, query } = await postAnalyticsBody({ cube: 'x', measures: ['count'] });

expect(res.statusCode).toBe(200);
expect(query).toHaveBeenCalledOnce();
});
});
4 changes: 3 additions & 1 deletion packages/runtime/src/dispatcher-validation-error.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -239,7 +239,9 @@ async function postAnalyticsQuery(err: unknown) {
expect(handler, 'POST /api/v1/analytics/query must be mounted').toBeTypeOf('function');

const res = makeRes();
await handler({ body: { cube: 'x', query: {} }, query: {} }, res);
// [#3878] Body must pass entry validation so the SERVICE's thrown error —
// the thing under test — is what reaches the exit, not an entry 400.
await handler({ body: { cube: 'x', measures: ['count'] }, query: {} }, res);
return res;
}

Expand Down
2 changes: 1 addition & 1 deletion packages/runtime/src/domain-handler-registry.test.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -135,7 +135,7 @@ describe('HttpDispatcher domain registry (D11 step ③)', () => {
analyticsQuery: vi.fn().mockResolvedValue({ rows: [{ n: 1 }] }),
};
const result = await makeDispatcher({ analytics }).dispatch(
'POST', '/analytics/query', { metric: 'count' }, {}, {} as any,
'POST', '/analytics/query', { cube: 'orders', measures: ['count'] }, {}, {} as any,
);
expect(result.handled).toBe(true);
// The bridge consulted the service (whichever entry point it uses).
Expand Down
62 changes: 62 additions & 0 deletions packages/runtime/src/domains/analytics.ts
Original file line numberDiff line numberDiff line change
Expand Up@@ -11,9 +11,65 @@
*/

import { CoreServiceName } from '@objectstack/spec/system';
import { AnalyticsQueryRequestSchema } from '@objectstack/spec/api';
import type { HttpProtocolContext, HttpDispatcherResult } from '../http-dispatcher.js';
import type { DomainHandlerDeps, DomainRoute } from '../domain-handler-registry.js';

/**
* [#3878] The duck-typed validation-failure shape both dispatcher error exits
* already map to 400 `VALIDATION_FAILED` + `details.fields[]` (see
* `validation-failure.ts`, #3918) — thrown here so entry validation needs no
* new error channel and no dependency on objectql's `ValidationError` class.
*/
function validationFailure(message: string, fields: unknown[]): Error {
const err = new Error(message) as Error & { code: string; fields: unknown[] };
err.name = 'ValidationError';
err.code = 'VALIDATION_FAILED';
err.fields = fields;
return err;
}

/**
* [#3878] Reject a malformed `AnalyticsQuery` body AT THE ENTRY with a 400
* naming what is wrong, instead of letting it reach the engine — where a
* shapeless body used to infer a column-less cube and die as an SQL syntax
* error deep in the driver (`SELECT FROM …`), or worse, have its off-contract
* filter key silently ignored.
*
* The retired `{ cube, query: {...} }` envelope (#3891 shim dialect) needs no
* special case here: the schema tombstones `query`/`format` (`retiredKey`),
* so the Zod issue itself carries the migration prescription. Only `filters` —
* never a declared key, so `.strict()` would answer a generic
* "unrecognized key" — gets a bespoke hint at the contract field `where`.
*
* Validation only — the ORIGINAL body is forwarded to the service untouched.
* (Parsing would inject the schema's `timezone: 'UTC'` default and silently
* override the engine's org-timezone resolution, #1982/#2018.)
*/
function assertAnalyticsQueryBody(body: unknown): void {
if (body && typeof body === 'object' && !Array.isArray(body)) {
const b = body as Record<string, unknown>;
if ('filters' in b && !('where' in b)) {
throw validationFailure(
'`filters` is not an AnalyticsQuery field — use `where` (canonical Query DSL FilterCondition, the same shape find() takes).',
[{ field: 'filters', code: 'unrecognized_keys', message: 'use `where` instead of `filters`' }],
);
}
}
const parsed = AnalyticsQueryRequestSchema.safeParse(body);
if (!parsed.success) {
const fields = parsed.error.issues.map((issue) => ({
field: issue.path.length > 0 ? issue.path.join('.') : '(body)',
code: issue.code,
message: issue.message,
}));
throw validationFailure(
`Invalid AnalyticsQuery body: ${fields.map((f) => `${f.field}: ${f.message}`).join('; ')}`,
fields,
);
}
}

export function createAnalyticsDomain(deps: DomainHandlerDeps): DomainRoute {
return {
prefix: '/analytics',
Expand All@@ -39,6 +95,10 @@ export async function handleAnalyticsRequest(

// POST /analytics/query
if (subPath === 'query' && m === 'POST') {
// [#3878] Entry validation AFTER the service check on purpose: an
// uninstalled analytics capability answers 404 (the honest "install
// service-analytics" signal, #3891) regardless of body shape.
assertAnalyticsQueryBody(body);
// [#2852] Pass the request's execution context so the analytics
// service scopes each object by its per-object read filter (tenant +
// RLS). Without it, `getReadScope(object, undefined)` returned no
Expand All@@ -60,6 +120,8 @@ export async function handleAnalyticsRequest(

// POST /analytics/sql (Dry-run or debug)
if (subPath === 'sql' && m === 'POST') {
// [#3878] Same body contract as /query — validated the same way.
assertAnalyticsQueryBody(body);
// [#2852] Scope the generated SQL to the caller too, so a preview
// reflects the same per-object read filter the real query applies.
const result = await analyticsService.generateSql(body, context?.executionContext);
Expand Down
Loading
Loading