fix(metadata-protocol,objectql)!: retire the degraded analytics shim — empty slot 404s instead of serving unscoped aggregates (#3891)

.changeset/retire-degraded-analytics-shim.md

@@ -0,0 +1,47 @@
+---
+"@objectstack/metadata-protocol": major
+"@objectstack/objectql": major
+---
+
+fix(metadata-protocol,objectql)!: retire the degraded analytics shim — the `analytics` slot stays empty without service-analytics (#3891, #3878)
+
+The protocol assembly (`assembleMetadataProtocol`, used by both
+`MetadataProtocolPlugin` and `ObjectQLPlugin`'s built-in mode) used to register
+a lightweight `analytics` fallback so `POST /api/v1/analytics/query` kept
+answering on installs without `@objectstack/service-analytics`. That fallback
+is **removed**, and with it the facade methods that existed only to serve it:
+`ObjectStackProtocolImplementation.analyticsQuery` / `getAnalyticsMeta` (the
+class no longer implements `AnalyticsProtocol`).
+
+Why removal instead of repair (#3891):
+
+- **It dropped the caller's ExecutionContext at the door.** The dispatcher
+ passes `context.executionContext` (#2852), but the shim's `query` was
+ unary — aggregation reached `engine.aggregate` with no context, the security
+ middleware's empty-principal branch waved it through, and **no RLS or tenant
+ predicate was injected**. An authenticated caller got a 200 with rows RLS
+ would hide.
+- **It ignored the contract filter.** `AnalyticsQuery`'s canonical filter field
+ is `where`; the shim read only a non-contract `filters` key, so a
+ spec-conformant filtered request silently returned a full-table aggregate.
+- **Every security gate had to be built twice** (#3770 on the shim vs
+ #3867/#3875 on the real engine) — the "duplicates logic only, harmless"
+ assessment in ADR-0076 D10 did not survive contact with reality.
+
+`getDiscovery()` stops hardcoding analytics as an always-on kernel service —
+the entry is now computed from the service registry like every other optional
+service (`enabled: false, status: 'unavailable'` and **no advertised route**
+when absent), which also removes the pre-#2462 discovery lie the shim was
+originally invented to make true.
+
+**Migration.** Deployments that relied on the fallback (programmatic
+`createStandaloneStack()` / `createObjectQLKernel()` embeds, hosts whose bundle
+doesn't require `analytics`): install `@objectstack/service-analytics` and
+mount `AnalyticsServicePlugin` — the real, context-aware engine. Without it,
+`/api/v1/analytics/*` now answers **404 ROUTE_NOT_FOUND** (previously: 200 with
+unscoped, unfiltered aggregates) and discovery reports
+`analytics: { enabled: false, status: 'unavailable' }`. Callers of
+`protocol.analyticsQuery(...)` / `protocol.getAnalyticsMeta(...)` must use the
+`analytics` service (`kernel.getService('analytics')`) instead. `os serve`
+default/full presets and managed environments already force the real engine and
+are unaffected.

content/docs/api/data-api.mdx

@@ -184,7 +184,10 @@ to set a new name or clear a unique field.
 
 ## Analytics
 
-Semantic BI queries using a cube-style API. Available when the analytics service is enabled.
+Semantic BI queries using a cube-style API. Provided by `@objectstack/service-analytics` —
+on deployments without it these endpoints answer `404 ROUTE_NOT_FOUND` and discovery
+reports `analytics: { enabled: false, status: "unavailable" }`. (The former kernel-level
+degraded fallback was retired — it served unscoped, unfiltered aggregates.)
 
 ### `POST /analytics/query`
 

content/docs/kernel/services-checklist.mdx

@@ -34,16 +34,16 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
 ┌─────────────────────────────────────────────────────────┐
 │ Kernel Layer │
 │ ┌──────────────────┐ ┌──────────────────────────────┐ │
-│ │ metadata (⚠️) │ │ data + analytics (✅) │ │
+│ │ metadata (⚠️) │ │ data (✅) │ │
 │ │ In-memory only │ │ ObjectQL example kernel │ │
 │ │ DB persistence │ │ Will be rebuilt as plugins │ │
 │ │ pending │ │ │ │
 │ └──────────────────┘ └──────────────────────────────┘ │
 ├─────────────────────────────────────────────────────────┤
 │ Plugin Layer │
-│ All other services: auth, automation, workflow, ui, │
-│ realtime, notification, ai, i18n, search,  │
-│ file-storage, cache, queue, job │
+│ All other services: analytics, auth, automation,  │
+│ workflow, ui, realtime, notification, ai, i18n, │
+│ search, file-storage, cache, queue, job │
 │ │
 │ Discovery API reports: enabled/unavailable/degraded │
 │ per service so clients adapt their UI accordingly │
@@ -58,7 +58,7 @@ The ObjectStack protocol defines **16 kernel services** registered via the `Core
 |:--|:-------------|:------------|:--------|:-------|:---------|
 | 1 | **metadata** | `core` | 7 | ⚠️ Framework | Kernel (in-memory) |
 | 2 | **data** | `required` | 9 | ✅ Implemented | `@objectstack/objectql` |
-| 3 | **analytics** | `optional` | 2 | ✅ Implemented | `@objectstack/objectql` |
+| 3 | **analytics** | `optional` | 2 | ❌ Plugin Required | `@objectstack/service-analytics` |
 | 4 | **auth** | `core` | — | 🟡 In Development | `@objectstack/plugin-auth` |
 | 5 | **ui** | `optional` | 5 | ❌ Plugin Required | TBD plugin |
 | 6 | **workflow** | `optional` | 3 | ❌ Plugin Required | TBD plugin |
@@ -173,23 +173,26 @@ The discovery endpoint returns a `services` map so clients know what is availabl
 
 ---
 
-## 3. analytics Service ✅ Implemented
+## 3. analytics Service — Plugin Required
 
 **Service Name**: `analytics` · **Criticality**: `optional` 
-**Implementation**: `@objectstack/objectql` (driver-level capability) 
-**Route Mount**: `/api/v1/analytics`
+**Implementation**: `@objectstack/service-analytics` (the only implementation) 
+**Route Mount**: `/api/v1/analytics` — only served when the plugin registers the service
 
-### Protocol Methods
-
-| Method | Signature | Status |
-|:-------|:----------|:------:|
-| `analyticsQuery` | `AnalyticsQueryRequest → AnalyticsResultResponse` | ✅ |
-| `getAnalyticsMeta` | `GetAnalyticsMetaRequest → AnalyticsMetadataResponse` | ✅ |
+<Callout type="warn">
+The kernel-level "degraded analytics fallback" (a lightweight ObjectQL adapter
+registered by the protocol assembly) was **retired** (#3891): it dropped the
+caller's `ExecutionContext` — aggregates ran without RLS/tenant scoping — and
+silently ignored the contract `where` filter. Without
+`@objectstack/service-analytics`, `/api/v1/analytics/*` now answers **404** and
+discovery reports `analytics: { enabled: false, status: "unavailable" }`.
+</Callout>
 
-### Features
+### Features (via `@objectstack/service-analytics`)
 
-- Cube semantic queries: measures / dimensions / filters → ObjectQL aggregation
-- Auto-generated metadata from SchemaRegistry (numeric → sum/avg, date → time dimensions)
+- Cube semantic queries: measures / dimensions / `where` filter → SQL or ObjectQL aggregation
+- Context-aware read scoping: per-object RLS/tenant predicates resolved from the request's `ExecutionContext` (fail-closed)
+- Cube metadata (`/analytics/meta`), SQL preview (`/analytics/sql`), dataset queries (`/analytics/dataset/query`)
 
 ---
 

docs/adr/0076-objectql-core-tiering.md

@@ -104,15 +104,15 @@ The segmentation is **spec/type-level and may start incrementally now** (define
 
 A single `ObjectStackProtocolImplementation` facade — and any `*-protocol` *implementation* package — is the wrong end-state. Verified against source:
 - The facade implements only **4 of the 11** contract domains (data, metadata, analytics, feed). The other 7 (realtime / notifications / workflow / ai / i18n / views / permissions) are **not implemented in it** — they belong to their own services or aren't implemented at all. *(Update #1959: the feed domain was retired per ADR-0052 §5 — `IFeedService`, `FeedProtocol`, and the facade's feed forwarding no longer exist, so the facade now implements 3 domains: data, metadata, analytics.)*
-- **Analytics uses a deliberate fallback + `replaceService` pattern (NOT a collision/bug — corrected in rev.9)**: `registerService` throws on duplicate (`core/kernel.ts`), so `ObjectQLPlugin` registers a lightweight ~66-line analytics **fallback** (so `/analytics` doesn't 404 in minimal deployments), and `AnalyticsServicePlugin`, when installed, calls **`ctx.replaceService('analytics', …)`** to swap in the full ~1.8k-LOC engine (three strategies incl. native-SQL). With service-analytics present the real engine serves (no shadowing); without it the fallback serves. The fallback duplicates *logic* only (a minor maintenance cost) and is intentional and harmless.
+- **Analytics uses a deliberate fallback + `replaceService` pattern (NOT a collision/bug — corrected in rev.9)**: `registerService` throws on duplicate (`core/kernel.ts`), so `ObjectQLPlugin` registers a lightweight ~66-line analytics **fallback** (so `/analytics` doesn't 404 in minimal deployments), and `AnalyticsServicePlugin`, when installed, calls **`ctx.replaceService('analytics', …)`** to swap in the full ~1.8k-LOC engine (three strategies incl. native-SQL). With service-analytics present the real engine serves (no shadowing); without it the fallback serves. The fallback duplicates *logic* only (a minor maintenance cost) and is intentional and harmless. *(Update #3891: "duplicates logic only / harmless" was refuted by evidence — the same security gate had to be built twice (#3770 for the fallback, #3867/#3875 for the real engine), and the fallback dropped the caller's ExecutionContext (aggregates ran without RLS/tenant predicates) and ignored the contract `where` filter. The fallback is **retired**: without service-analytics the `analytics` slot stays empty, `/analytics` answers the dispatcher's existing 404, and discovery reports `unavailable` — the D12 machinery makes the empty slot honest, which removes the fallback's original reason to exist (the discovery hardcode).)*
 - ~~**Feed is already delegated** to `IFeedService`; the facade only forwards.~~ *(Retired per ADR-0052 §5 / #1959 — `IFeedService` and the facade's feed forwarding were removed.)*
 - The domain service packages already exist: `service-analytics`, `service-messaging`, `service-realtime`.
 
 Target end-state:
 - **"protocol" names ONLY the contract** — the segmented interfaces in `@objectstack/spec/api` (D9). There is **no `*-protocol` implementation package**.
 - **`DataProtocol`** impl → engine-adjacent / transport (thin wire-normalizers).
 - **`MetadataProtocol`** impl → stays in **`@objectstack/metadata-protocol`** (name **retained** — already published; renaming churns downstream for ~0 benefit). The package's *content* converges to the metadata-management impl (it owns `sys_metadata`). The `protocol` in the name is a deliberate, low-cost naming exception from being published — the real contract lives in `@objectstack/spec/api`, not here. (Open Question #7, resolved.)
-- **Analytics / Realtime / Notification / …** → each domain's *full* impl lives in its existing service package. For analytics specifically, the `ObjectQLPlugin` fallback **must be preserved or consciously dropped** (it prevents `/analytics` 404 for deployments without service-analytics) — **not blindly deleted**. The engine keeps only the minimal fallback it deliberately provides. *(Feed was dropped entirely per ADR-0052 §5 / #1959 — no service package, no facade forwarding.)*
+- **Analytics / Realtime / Notification / …** → each domain's *full* impl lives in its existing service package. For analytics specifically, the `ObjectQLPlugin` fallback **must be preserved or consciously dropped** (it prevents `/analytics` 404 for deployments without service-analytics) — **not blindly deleted**. The engine keeps only the minimal fallback it deliberately provides. *(Feed was dropped entirely per ADR-0052 §5 / #1959 — no service package, no facade forwarding.)* *(Update #3891: consciously dropped — see the fallback bullet above. The facade's `analyticsQuery`/`getAnalyticsMeta` went with it, so the facade now implements 2 domains (data, metadata) and analytics has exactly one implementation, in service-analytics — this bullet's end-state for the domain.)*
 - **The transport/dispatcher routes each contract-slice to the owning service** (it already resolves services by name) — no central facade class.
 
 This **refines D1** (the `metadata-protocol` package was an intermediate, not the end-state) and **completes D9**. Executed at the cross-repo window with D7 / Step 2.
@@ -143,7 +143,7 @@ Decision: each capability plugin registers its routes as a **normalized handler*
 
 This fixes the **whole class at once — without deleting any fallback** (no `/analytics` 404 regression): the analytics fallback and the dev stubs simply stop *lying*; they keep serving but are honestly labelled. It is the runtime enforcement of the D9-refinement principle (capabilities = what is actually installed, computed at runtime).
 
-**Supersedes the rev.9 analytics conclusion**: the fix for the analytics fallback is to **mark it honestly (this D12)**, not "preserve-or-delete".
+**Supersedes the rev.9 analytics conclusion**: the fix for the analytics fallback is to **mark it honestly (this D12)**, not "preserve-or-delete". *(Update #3891: superseded in turn for the analytics fallback specifically — honest labelling was necessary but not sufficient. The fallback's `degraded` label was accurate about capability, yet nothing in it disclosed that aggregates ran WITHOUT the caller's RLS/tenant scoping and that the contract `where` filter was ignored; an authorized caller still got a 200 with wrong (over-broad) numbers. A fallback may degrade features, never security semantics — so it was retired, and the "no `/analytics` 404 regression" goal above is deliberately abandoned for this slot: the 404 IS the honest signal. D12's marker/discovery machinery stays, and is what makes the now-empty slot report `unavailable`.)*
 
 **Execution**: framework (marker convention + `svcAvailable` respects it + discovery schema `stub` status) and console (read the honest status) land **together at the cross-repo window** — the console reads `discovery.services`, so this is a cross-repo contract change.
 

packages/metadata-protocol/src/plugin.ts

@@ -5,8 +5,9 @@
  *
  * Owns what `ObjectQLPlugin` historically assembled inline: the
  * `ObjectStackProtocolImplementation` construction + `protocol` service
- * registration, the metadata-storage platform objects, and the lightweight
- * `analytics` fallback. Registering it NEXT TO `ObjectQLPlugin` (with
+ * registration and the metadata-storage platform objects. (The lightweight
+ * `analytics` fallback that used to ride here was retired in #3891 — see
+ * {@link assembleMetadataProtocol}.) Registering it NEXT TO `ObjectQLPlugin` (with
  * `registerProtocol: false` on the engine plugin) makes `@objectstack/objectql`
  * effectively protocol-free at boot-assembly level — the engine plugin keeps
  * only protocol CONSUMERS (DB hydration + authored hook/action rebind, both of
@@ -22,7 +23,6 @@
  */
 
 import type { Plugin, PluginContext } from '@objectstack/core';
-import { SERVICE_SELF_INFO_KEY, type ServiceSelfInfo } from '@objectstack/spec/api';
 import {
  SysMetadataObject,
  SysMetadataHistoryObject,
@@ -73,12 +73,19 @@ export function createMetadataProtocolPlugin(options: MetadataProtocolPluginOpti
 
 /**
  * The ONE protocol assembly (ADR-0076 Step 2 PR-C): metadata-storage platform
- * objects + `ObjectStackProtocolImplementation` as the `protocol` service +
- * the D12 `degraded` analytics fallback. Called by
- * {@link createMetadataProtocolPlugin} (delegated mode) AND by
+ * objects + `ObjectStackProtocolImplementation` as the `protocol` service.
+ * Called by {@link createMetadataProtocolPlugin} (delegated mode) AND by
  * `ObjectQLPlugin`'s built-in convenience mode (`registerProtocol !== false`)
  * — single source, two mounts, identical result.
  *
+ * The `analytics` slot is deliberately NOT filled here (#3891 / #3878,
+ * superseding the ADR-0076 D10/D12 "preserve the fallback" stance): the
+ * degraded shim dropped the caller's ExecutionContext (no RLS/tenant
+ * predicates) and ignored the contract `where` filter, returning full-table
+ * aggregates with a 200. An empty slot degrades honestly instead — the
+ * dispatcher's `/analytics` domain answers 404 and discovery reports
+ * `unavailable` until `AnalyticsServicePlugin` registers the real engine.
+ *
  * @returns the protocol shim, so the engine-side caller can arm its
  * mutation-rebind subscription synchronously.
  */
@@ -119,42 +126,13 @@ export function assembleMetadataProtocol(
  ctx.registerService('protocol', protocolShim);
  ctx.logger.info('Protocol service registered (MetadataProtocolPlugin)');
 
- // Lightweight `analytics` fallback mapped onto the protocol shim's
- // `analyticsQuery` — kept with the protocol assembly so `/analytics`
- // keeps answering on installs without service-analytics. Honest
- // capabilities (ADR-0076 D12): self-identifies as `degraded`;
- // AnalyticsServicePlugin replaces it (ctx.replaceService) with the
- // real engine.
- ctx.registerService('analytics', {
- [SERVICE_SELF_INFO_KEY]: {
- status: 'degraded',
- handlerReady: true,
- message: 'Lightweight ObjectQL analytics fallback — install @objectstack/service-analytics for the full engine',
- } satisfies ServiceSelfInfo,
- // HttpDispatcher passes the raw POST body (AnalyticsQuery
- // shape); the shim's `analyticsQuery` expects the wrapped
- // `{ cube, query }` envelope and returns its own `{ success,
- // data }` — reshape in, unwrap out (one level), exactly as the
- // historical inline adapter did.
- query: async (body: any) => {
- const envelope = body && typeof body === 'object' && 'query' in body && 'cube' in body
- ? body
- : { cube: body?.cube, query: body };
- const result = await protocolShim.analyticsQuery(envelope);
- if (result && typeof result === 'object' && 'success' in result && 'data' in result) {
- return (result as any).data;
- }
- return result;
- },
- getMeta: async () => ({
- cubes: [],
- message: 'Analytics meta endpoint not implemented by ObjectQL adapter',
- }),
- generateSql: async (_body: any) => ({
- sql: null,
- message: 'Analytics SQL generation not implemented by ObjectQL adapter',
- }),
- });
+ // NO `analytics` fallback rides here anymore (#3891 / #3878). The
+ // degraded shim this assembly used to register dropped the request's
+ // ExecutionContext (aggregates ran without RLS/tenant predicates)
+ // and ignored the contract filter field `where` — a 200 with wrong
+ // numbers. The slot now stays empty: `/analytics/*` answers 404
+ // (ROUTE_NOT_FOUND) and discovery reports the service unavailable
+ // until @objectstack/service-analytics registers the real engine.
 
  return protocolShim;
 }