From 583b528c2d6fbbc7a7e4f468aedb16d0cb99b34b Mon Sep 17 00:00:00 2001 From: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Date: Sun, 31 May 2026 23:49:01 +0800 Subject: [PATCH 1/3] =?UTF-8?q?docs(adr-0021):=20revise=20after=20implemen?= =?UTF-8?q?tation=20scan=20=E2=80=94=20engine=20join=20gap,=20Cube=20recon?= =?UTF-8?q?ciliation,=20naming,=20task=20list?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A four-area code scan (spec/runtime/frontend/examples) found three things that change the plan: - Runtime != schema: groupBy/aggregations execute (single-object), but joins/having/windowFunctions are schema-only and NOT executed by IDataEngine or the SQL driver. The headline "revenue by account.region" cannot run via IDataEngine today — joins are the gating gap. - A parallel semantic layer already exists and is implemented (IAnalyticsService + CubeSchema); its NativeSQLStrategy is the only cross-object JOIN path but bypasses RLS/tenant. Don't build a third layer — reuse/compile-to Cube. - Naming collisions: Dataset/Dimension/Metric are already taken. Corrects the false "engine already does joins" premise, adds the Cube-reconciliation + naming decisions (D-A/B/C), and appends an implementation-scan section with a workstream task list and rough sizing (~6-8 weeks across both repos; rendering lives in sibling repo objectui). --- .../0021-analytics-dataset-semantic-layer.md | 72 ++++++++++++++++++- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/docs/adr/0021-analytics-dataset-semantic-layer.md b/docs/adr/0021-analytics-dataset-semantic-layer.md index 6fd3eab8ad..697547ea8d 100644 --- a/docs/adr/0021-analytics-dataset-semantic-layer.md +++ b/docs/adr/0021-analytics-dataset-semantic-layer.md @@ -1,6 +1,6 @@ # ADR-0021: Analytics — one semantic `dataset` layer, `report` / `dashboard` become pure presentation -**Status**: Proposed (2026-05-31) +**Status**: Proposed — **revised 2026-05-31 after an implementation scan** (see "Implementation scan" near the end; the original "the engine already does joins" premise was corrected) **Deciders**: ObjectStack Protocol Architects **Builds on**: [ADR-0019](./0019-app-as-consumer-unit.md) + [ADR-0020](./0020-state-machine-converge-and-enforce.md) (the "one engine, fold the parasitic concept into its host" principle — applied here to analytics), [ADR-0017](./0017-object-has-many-view.md) (object-bound ListView is the row-level lens), [ADR-0010](./0010-nl-to-flow-authoring.md) + [ADR-0011](./0011-actions-as-ai-tools.md) (AI authoring is the design center) **Consumers**: `@objectstack/spec` (`ui/report.zod.ts`, `ui/dashboard.zod.ts`, `ui/view.zod.ts`, `data/query.zod.ts`, `kernel/metadata-type-schemas.ts`), `@objectstack/objectql` (query engine), `@objectstack/analytics-service`, all `examples/*` @@ -11,7 +11,9 @@ ## TL;DR -The platform already owns a **complete query engine** — `QuerySchema` has `joins` (inner/left/right/full + strategies + subquery + cross-datasource), `aggregations`, `groupBy` (with date bucketing), `having`, and `windowFunctions` ([`query.zod.ts:586`](../../packages/spec/src/data/query.zod.ts#L586)). +The platform's **query *schema*** (`QuerySchema`, [`query.zod.ts:586`](../../packages/spec/src/data/query.zod.ts#L586)) already *describes* `joins` (inner/left/right/full + strategies + subquery + cross-datasource), `aggregations`, `groupBy` (with date bucketing), `having`, and `windowFunctions`. + +> **Correction (revised 2026-05-31).** An implementation scan found the *runtime* does **not** match the schema: `groupBy` + `aggregations` execute (single-object only), but `joins` / `having` / `windowFunctions` are **schema-only — not executed** by `IDataEngine` or the SQL driver. A separate, already-implemented **Cube semantic layer** (`IAnalyticsService` + `CubeSchema`) is the *only* path that emits cross-object joins today (and it bypasses RLS/tenant). This reframes the work and the naming — see the "Implementation scan" section. The decisions below stand; the cost and the build-vs-reuse choice change. But **three presentation surfaces each re-implement their own crippled, single-object query inline** instead of using it: @@ -299,6 +301,72 @@ Duplicate inline definitions that the codemod detects as identical (same object+ --- +## Implementation scan (revised 2026-05-31) — reality, gaps, task list + +A four-area code scan (spec, runtime, frontend, examples) produced the following. **Three findings change the plan; read them before estimating.** + +### Finding 1 — the runtime does not match `QuerySchema` + +| Capability | Schema | Runtime reality | +|---|---|---| +| `groupBy` + `aggregations` | ✅ | ✅ executed — `in-memory-aggregation.ts` + `driver-sql` groupBy; **single-object only** | +| `joins` | ✅ | ❌ **not executed** — `engine.find/aggregate` never read `joins`; SQL driver `find`/`aggregate` emit no JOIN. Cross-object is only FK-expand (`expandRelatedRecords`, N+1, *nested* — not a flat join, unusable for grouped aggregation) | +| `having` | ✅ | ❌ not evaluated | +| `windowFunctions` | ✅ | ❌ dead code (`findWithWindowFunctions` never called by `ObjectQL`) | + +So "revenue by `account.region`" — the headline dataset use case — **cannot run through `IDataEngine` today.** Joins are the gating runtime gap. + +### Finding 2 — a parallel semantic layer already exists (must reconcile) + +`data/analytics.zod.ts` `CubeSchema` + `contracts/analytics-service.ts` `IAnalyticsService` are **implemented** (`AnalyticsService` in `service-analytics`, `MemoryAnalyticsService` in `driver-memory`): a Cube.io-style `{ measures, dimensions, timeDimensions }` layer — conceptually the same "semantic layer" this ADR proposes. Its `NativeSQLStrategy` is the **only** code that emits cross-object `LEFT JOIN` (single-hop), but via raw `engine.execute()` which **bypasses the sharing-middleware RLS and tenant isolation** ([`engine.ts:2077`](../../packages/objectql/src/engine.ts#L2077) warns explicitly). It loads opt-in (`requires: ['analytics']`), and its grammar is **disjoint from `QuerySchema`** — nothing compiles `QuerySchema.joins/having/window` to execution. + +**Implication:** do not build a *third* semantic layer. Either (b) adopt/extend Cube as the dataset, or (c) compile `dataset` → `AnalyticsQuery` and reuse the Cube runtime (then harden its RLS/tenant). Both are far cheaper than teaching `IDataEngine` to join. + +### Finding 3 — naming collisions + +The ADR's `DatasetSchema` / `DimensionSchema` / `MeasureSchema` are all taken: `DatasetSchema` = seed data (`data/dataset.zod.ts`, widely consumed); `DimensionSchema` + `MetricSchema` = the existing Cube layer (`data/analytics.zod.ts`). The new type must be renamed or namespaced (e.g. `AnalyticsDataset` / `Measure`). **The ADR's literal names cannot land as-is.** + +### Other scan facts that size the work + +- **Rendering is in the sibling repo `objectui`**, not here — Dashboard/Report renderers, `useReportData`, builders, the `data-objectstack` adapter. Any field change is a **two-repo change** + a `.objectui-sha` bump. +- **`ReportSchema` (pivot) has no runtime executor** — `plugin-reports` is a separate saved-query/CSV emailer; pivoting happens entirely client-side in `useReportData.ts`. +- **Registration touches 4 surfaces**, not 1: `kernel/metadata-type-schemas.ts`, `kernel/metadata-plugin.zod.ts` (enum + `DEFAULT_METADATA_TYPE_REGISTRY`, `loadOrder` < report/dashboard), `shared/metadata-collection.zod.ts` (`MAP_SUPPORTED_FIELDS` + `PLURAL_TO_SINGULAR`), `objectql/engine.ts` `metadataArrayKeys` (**two** lists, L806 + L961). +- **Migration scope:** 7 reports (2 files) + 64 widgets (4 files; heaviest `chart-gallery.dashboard.ts` = 38) + 2 chart-views (2 files) = **8 source files, two inline shapes**. Tests to rewrite: `view.test.ts` (214) + `dashboard.test.ts` (146) + `report.test.ts` (51) + `report-service.test.ts` + `view-expand.test.ts` + 3 example integration tests. No JSON/seed instance data. JSON-schema regenerates via `pnpm gen:schema`. + +### Pre-work decisions (gate everything) + +| # | Decision | Options | Recommendation | +|---|---|---|---| +| D-A | Relationship to the existing Cube layer | (a) new type wrapping `QuerySchema`, build joins into the engine; (b) dataset **is** Cube, renamed/extended; (c) dataset **compiles to** `AnalyticsQuery`, reuse Cube runtime | **(c) / (b)** — reuse the implemented, join-capable Cube runtime; don't build a third layer | +| D-B | Naming collisions | rename (`AnalyticsDataset`/`Measure`) or namespace | rename; literal ADR names can't land | +| D-C | Join execution + safety | build join/having into `IDataEngine`+SQL driver; **or** route via Cube `NativeSQLStrategy` **and add RLS + tenant scoping** to that path | reuse Cube path **+ harden RLS/tenant** (enterprise red line) | + +### Task list (workstreams; S≈½d, M≈1–3d, L≈1wk, XL≈2wk+; ⛔ = blocked by D-A/B/C) + +**WS0 · Design close-out (blocks all) — M** +- ⛔ Resolve D-A / D-B / D-C; re-merge this ADR with the chosen direction. + +**WS1 · Spec / Schema (this repo) — L** +- `ui/dataset.zod.ts` (resolved names; dimensions/measures; `query` *or* mapping to Cube). +- Refactor `report.zod.ts` (→ `dataset`/`rows`/`columns`/`values`/`sections`), `dashboard.zod.ts` (widget → `dataset`/`report`/`dimensions`/`measures`/`viz`/`drillTo` + `superRefine`; globalFilters/dateRange → dimension names), `view.zod.ts` (drop `ListChartConfigSchema` + `type:'chart'`). +- Wire the 4 registration surfaces (above). Rewrite `report.form.ts`/`dashboard.form.ts` + add `dataset.form.ts`. Adjust `analytics-service`/`report-service`/`ui-service` contracts. Run `pnpm gen:schema`. + +**WS2 · Runtime / engine (this repo) — XL (bottleneck)** +- ⛔ Join execution (engine join/having, or route dataset → Cube `NativeSQLStrategy`). +- ⛔ Add RLS + tenant safety to the join path. +- Dataset executor (dataset + selected dims/measures + runtimeFilter/compareTo → executable query → chart data); server-side `compareTo` time-shift; dataset metadata loading. + +**WS3 · Frontend + designers (sibling repo `objectui`, cross-repo) — XL** +- Rework `DashboardRenderer.getComponentSchema()` and `useReportData.ts` to bind to dataset+measures; adapt `data-objectstack` `aggregate()`; convert `WidgetConfigPanel`/`DashboardEditor`/`ReportDesigner`/`ReportConfigPanel` from field pickers to dataset+measure pickers; new dataset editor + picker; bump `.objectui-sha`. + +**WS4 · Migration codemod + tests (this repo) — L** +- Codemod (two inline shapes) over 8 files; hoist duplicate inline definitions into named datasets. Rewrite the listed tests + barrels/`setup.app.ts`. + +**WS5 · Docs / skills (this repo) — M** +- ~9 mdx pages + `meta.json` nav; `skills/objectstack-ui/SKILL.md` (major) + `objectstack-query` (minor). + +**Critical path:** WS0 → (WS1 ∥ WS2) → WS3 → WS4. **Rough order of magnitude:** ~6–8 weeks for one senior engineer across both repos; choosing D-A = (c) reuse Cube roughly halves WS2. + ## Open questions 1. **Calculated/derived measures** (e.g. `win_rate = won / total`) — introduce a `MeasureSchema.expression` (measure-over-measures) now, or defer to a follow-up ADR? Leaning: define the field now, implement later. From 8bc9cf221e8f3e20266c601090f37e4ae1c0e26d Mon Sep 17 00:00:00 2001 From: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Date: Sun, 31 May 2026 23:56:28 +0800 Subject: [PATCH 2/3] feat(connector-slack): Slack Web API connector + connector/channel reconciliation (ADR-0022) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add ADR-0022 reconciling the two seams that both claim 'Slack': Connector (integration mechanism, built) vs MessagingChannel (human-notification layer, ADR-0012/0013, draft). Decision: notify-a-human => MessagingChannel; raw API call => connector_action; the channel delegates its transport to a Connector. Back-references added to ADR-0012 §2 and ADR-0013 §9. Add @objectstack/connector-slack — the second reference concrete connector (after connector-rest), validating the baseline connector registry and opening the ADR-0022 'raw API call' path. Static bot-token (bearer) auth only; OAuth2 install/refresh and credential vaulting stay enterprise. Actions: chat.postMessage, chat.update, and a generic call escape hatch. Surfaces Slack's logical 'ok' (HTTP 200 even on failure) without throwing. Tests: 8/8 (connector unit + end-to-end kernel boot dispatching a connector_action flow to slack.chat.postMessage). --- docs/adr/0012-notification-platform.md | 2 + docs/adr/0013-bidirectional-messaging.md | 2 + .../0022-connectors-vs-messaging-channels.md | 150 +++++++++++++ packages/connectors/connector-slack/LICENSE | 93 ++++++++ .../connectors/connector-slack/package.json | 36 +++ .../src/connector-slack-plugin.test.ts | 78 +++++++ .../src/connector-slack-plugin.ts | 79 +++++++ .../connectors/connector-slack/src/index.ts | 28 +++ .../src/slack-connector.test.ts | 132 +++++++++++ .../connector-slack/src/slack-connector.ts | 210 ++++++++++++++++++ .../connectors/connector-slack/tsconfig.json | 10 + pnpm-lock.yaml | 22 ++ 12 files changed, 842 insertions(+) create mode 100644 docs/adr/0022-connectors-vs-messaging-channels.md create mode 100644 packages/connectors/connector-slack/LICENSE create mode 100644 packages/connectors/connector-slack/package.json create mode 100644 packages/connectors/connector-slack/src/connector-slack-plugin.test.ts create mode 100644 packages/connectors/connector-slack/src/connector-slack-plugin.ts create mode 100644 packages/connectors/connector-slack/src/index.ts create mode 100644 packages/connectors/connector-slack/src/slack-connector.test.ts create mode 100644 packages/connectors/connector-slack/src/slack-connector.ts create mode 100644 packages/connectors/connector-slack/tsconfig.json diff --git a/docs/adr/0012-notification-platform.md b/docs/adr/0012-notification-platform.md index ddeed3baa6..796ccb2522 100644 --- a/docs/adr/0012-notification-platform.md +++ b/docs/adr/0012-notification-platform.md @@ -180,6 +180,8 @@ Every layer has a stable seam. A custom channel (e.g. `plugin-notification-feish ### 2. The `MessagingChannel` interface +> **Transport delegation (ADR-0022).** Per [ADR-0022](./0022-connectors-vs-messaging-channels.md), a channel's *transport* — provider auth, base URL, rate-limit handling, the provider's action set — should be implemented on top of a `Connector` (ADR-0015/0018), not hand-rolled per channel. `MessagingChannel` adds only the messaging *semantics* (preferences, inbox, outbox, sessions) on that substrate. The interface below is unchanged; this is an implementation guideline for concrete channels. + ```ts // packages/spec/src/notification/channel.zod.ts export interface MessagingChannel { diff --git a/docs/adr/0013-bidirectional-messaging.md b/docs/adr/0013-bidirectional-messaging.md index 6bd30c583e..1f43a34ccf 100644 --- a/docs/adr/0013-bidirectional-messaging.md +++ b/docs/adr/0013-bidirectional-messaging.md @@ -483,6 +483,8 @@ The synthetic-user fallback is what keeps the system honest: nothing happens "as ### 9. Slack channel — first reference implementation +> **Transport delegation (ADR-0022).** The Slack channel's outbound `send()` (and any Web-API calls below) should delegate to a Slack `Connector` (e.g. `@objectstack/connector-slack`) for auth / base URL / rate-limit / the `chat.*` action set, rather than hand-rolling `fetch`. The channel adds the messaging semantics on top. See [ADR-0022](./0022-connectors-vs-messaging-channels.md) — this lets the `connector_action`-direct path and the channel share one Slack transport. + `@objectstack/plugin-messaging-slack`. Package layout: ``` diff --git a/docs/adr/0022-connectors-vs-messaging-channels.md b/docs/adr/0022-connectors-vs-messaging-channels.md new file mode 100644 index 0000000000..9929114f9a --- /dev/null +++ b/docs/adr/0022-connectors-vs-messaging-channels.md @@ -0,0 +1,150 @@ +# ADR-0022: Connectors vs Messaging Channels — Where "Push to Slack" Lives + +**Status**: Proposed (2026-05-31) +**Deciders**: ObjectStack Protocol Architects +**Builds on**: [ADR-0012](./0012-notification-platform.md) (Messaging Platform — outbound `notify` on a generalized outbox), [ADR-0013](./0013-bidirectional-messaging.md) (Bidirectional Messaging — Slack first), [ADR-0015](./0015-external-datasource-federation.md) (open mechanism / enterprise lifecycle split), [ADR-0018](./0018-unified-node-action-registry.md) (`connector_action` as baseline generic dispatch) +**Consumers**: `@objectstack/spec` (`integration/`, future `messaging/`), `@objectstack/services/service-automation` (connector registry), future `@objectstack/service-messaging`, future `@objectstack/plugin-messaging-slack`, `@objectstack/connectors/*` + +--- + +## TL;DR + +The question "should pushing a message to Slack be a **connector**?" surfaces a real collision in the repo: **Slack now has two candidate homes.** + +- **ADR-0012/0013** designate Slack as a **`MessagingChannel`** — a human-facing notification + conversation surface with a preference matrix, an inbox, an outbox, sessions/threads, and an inbound receiver. *Both ADRs are Draft; none of `service-messaging`, the `notify` transport, or `plugin-messaging-slack` is built yet.* +- **ADR-0015/0018 + the just-merged connector baseline** give us a **`Connector`** — a generic "call any external system's API" mechanism (`connector_action` node, `engine.registerConnector()`, `@objectstack/connector-rest`). *This is built and live.* + +This ADR resolves the collision instead of letting two seams quietly grow a second Slack integration. The decision: + +1. **A `Connector` is the transport/integration mechanism** — "talk to system X's API," no recipient semantics. A `MessagingChannel` is the **human-messaging semantic layer** — recipients, preferences, inbox, outbox, sessions, inbound. They are **different abstractions at different layers**, exactly as "SMTP transport" ≠ "send a notification" in ADR-0012. +2. **"Notify a human via Slack" is NOT a connector** — it is a `MessagingChannel` (ADR-0012/0013). Modeling it as a bare `connector_action` would silently drop preferences, the inbox, retry/dead-letter, and the reply-in-thread story, and would **duplicate ADR-0012**. +3. **"A flow makes a raw Slack API call" IS a connector action** — `connector_action(connectorId:'slack', actionId:'chat.postMessage', …)`. This is the generic-integration path and works on **today's** baseline with no new abstraction. +4. **The two share one transport.** A Slack `MessagingChannel.send()` should be **implemented on top of** a Slack `Connector` (auth, base URL, rate-limit, action set live once, in the connector); the channel adds the messaging semantics on top. Connector = mechanism; channel = policy. This generalizes ADR-0012's own "email becomes a transport sub-system" pattern. + +Because ADR-0012/0013 are still Draft, folding the connector baseline in underneath them is a **cheap revision, not a migration**. + +--- + +## Context + +### The two seams, side by side + +| Dimension | `Connector` (ADR-0015/0018, **built**) | `MessagingChannel` (ADR-0012/0013, **draft, unbuilt**) | +|:---|:---|:---| +| Purpose | call any external system's API | deliver to / converse with a **human** | +| Primitive | an **action** (a verb on a system) | a **channel** (a delivery medium for a person) | +| Recipient model | none — caller supplies explicit inputs | per-user preference matrix, mute, quiet hours, recipient resolver | +| Delivery guarantee | one call → success/fail | outbox, retry, dead-letter, cluster-lock | +| Threads / sessions | none | `sessionKey`, reply-to-thread, `sys_message_mirror` | +| Inbound | generic webhook trigger | verify+dedup+parse → `target-resolver` → flow / hook / AI tool | +| Multi-account | not modeled | `sys_channel_account`, first-class, multi-tenant | +| Identity mapping | none | `sys_channel_user_link` (Slack user ↔ `sys_user`) | +| Status today | **shipped** (`connector_action`, `@objectstack/connector-rest`) | **Draft** (`notify` is a no-op; `service-messaging` does not exist) | + +They overlap on exactly one thing: the bytes on the wire eventually hit Slack's `chat.postMessage`. Everything *around* that call is different — and that difference is the whole reason ADR-0012/0013 exist. + +### Why this is live now + +The connector baseline landed *after* ADR-0012/0013 were drafted. When 0013 was written, a Slack channel plugin was expected to hand-roll its own `fetch`, auth, base URL, and rate-limit handling ([0013 §9](./0013-bidirectional-messaging.md)). We now have a baseline that provides exactly those things generically. So the honest re-evaluation is twofold: + +1. **Routing question** — when a builder wants to "push to Slack," which seam do they reach for? +2. **Substrate question** — now that connectors are baseline, should the (still-unbuilt) Slack channel be re-specified to *sit on top of* a connector rather than hand-roll transport? + +### The precedent: this is the email split, generalized + +ADR-0012 already drew this exact line for email: SMTP/Resend/Postmark are **transports** (`EmailTransport`), and "send a notification" is a **channel** (`MessagingChannel`) that *uses* a transport and adds preferences + outbox + suppression on top. `Connector` is simply the *general* form of "transport": a uniform way to talk to any external API. So the reconciliation is not a new idea — it is applying ADR-0012's own layering to every channel, with `Connector` as the shared transport substrate. + +--- + +## Decision + +### 1. Three concerns, three homes — do not collapse them + +``` +┌────────────────────────────────────────────────────────────────────────┐ +│ CONVERSATION / INBOUND ADR-0013 │ +│ receiver · verify · sessions · target-resolver → flow/hook/AI tool │ +└───────────────────────────────┬──────────────────────────────────────────┘ + │ reuses +┌───────────────────────────────▼──────────────────────────────────────────┐ +│ HUMAN NOTIFICATION ADR-0012 │ +│ notify node · recipient resolve · preference matrix · inbox · outbox │ +│ MessagingChannel.send() │ +└───────────────────────────────┬──────────────────────────────────────────┘ + │ delegates transport to +┌───────────────────────────────▼──────────────────────────────────────────┐ +│ INTEGRATION MECHANISM ADR-0015 / 0018 (BUILT) │ +│ Connector · connector_action · auth · base URL · rate-limit · actions │ +└────────────────────────────────────────────────────────────────────────────┘ +``` + +- **Integration mechanism** (`Connector`, `connector_action`): "talk to any external API." Baseline, shipped. Slack-as-an-API-target is a legitimate connector. +- **Human notification** (`MessagingChannel`, `notify`): "tell a person, honoring their preferences, with durable delivery." ADR-0012. **A connector is not a substitute** for this layer. +- **Conversation/inbound** (sessions, resolvers, AI): ADR-0013. Built on the notification layer. + +### 2. Routing rule — answer to "should push-to-Slack be a connector?" + +| What the builder wants | Seam | Available | +|:---|:---|:---| +| "Notify the deal **owner** that the deal closed" (respect their channel preference, write the inbox, retry on failure, reply in the same thread) | **`MessagingChannel` + `notify` node** | when ADR-0012 ships | +| "Post **this exact text** to **#ops**" / "look up a Slack user" / "create a channel" — a raw API step with no recipient/preference semantics | **`connector_action` on a Slack `Connector`** | **today** (needs a `connector-slack` plugin) | + +The litmus test: **does a *person's preference* decide whether and where this message lands?** If yes → `MessagingChannel`. If the flow author has already decided the exact destination and just needs the API called → `connector_action`. + +### 3. One transport — the channel delegates to the connector + +A Slack `MessagingChannel.send()` SHOULD be implemented on top of a Slack `Connector`'s outbound action(s), so Slack auth, base URL, rate-limit handling, and the action set live in **one** place (the connector) and the channel adds only the messaging semantics (preference check, render, outbox row, session/thread decoding). This: + +- removes the hand-rolled `fetch` that [0013 §9](./0013-bidirectional-messaging.md) currently assumes; +- gives the connector path and the channel path a **single** Slack-auth/rate-limit implementation; +- lets the same Slack `Connector` serve raw `connector_action` flow steps *and* back the channel. + +This is the ADR-0012 "email = transport sub-system behind a channel" pattern, with `Connector` as the universal transport. + +### 4. Open-source / enterprise boundary (consistent with ADR-0015) + +| Capability | Tier | +|:---|:---| +| `Connector` contract + `connector_action` dispatch + in-process registry | **open** (shipped) | +| Static-auth REST connector (`@objectstack/connector-rest`) | **open** (shipped) | +| `MessagingChannel` interface + outbox + inbox + preference matrix + `notify` | **open baseline** — this is the #1292 P0 fix; it must be in the open framework | +| A community Slack connector / channel using a **static bot token** (bearer) | **open** | +| Managed OAuth2 **install + token refresh**, credential vault (`service-secrets`), multi-tenant `sys_channel_account` lifecycle, premium connectors, marketplace | **enterprise** (per ADR-0015; ADR-0013 leans on `service-secrets` + OAuth — those are the enterprise bits) | + +A static Slack bot token is just a bearer credential, so a basic open-source Slack connector + channel is shippable in the framework; the managed-credential / OAuth-refresh / multi-tenant lifecycle is the enterprise line — the same split ADR-0015 drew for datasources and ADR-0018's addendum drew for connectors. + +### 5. Anti-patterns this rules out + +- ❌ **A generic "notification connector."** Notification is a *semantic layer above* connectors, not a connector. Do not invent one. +- ❌ **Growing preference/inbox/outbox/session logic onto `Connector`.** That re-implements ADR-0012 in the wrong layer. The connector stays a dumb, generic API caller. +- ❌ **A Slack channel plugin that hand-rolls its own `fetch`/auth** when a Slack connector already encapsulates it. One transport implementation, reused. +- ❌ **Forcing `plugin-email` to become a connector right now.** It predates this, has its own template/transport story, and ADR-0012 already folds it in as `plugin-notification-email`. Leave it; revisit only if a generic transport buys something concrete. + +--- + +## Consequences + +**Positive** +- One unambiguous answer to "where does push-to-Slack live," with a one-line litmus test builders can apply. +- The unbuilt messaging stack gets a free, uniform transport substrate (the connector baseline) instead of N hand-rolled HTTP clients. +- The open/enterprise line stays identical across datasources, connectors, and channels — one mental model. +- No migration: ADR-0012/0013 are Draft, so this is a spec revision before code exists. + +**Negative / costs** +- ADR-0012/0013 need a small revision to say "the Slack channel's `send()` delegates to a Slack connector" (this ADR records the intent; the prose edits to 0012/0013 are follow-ups). +- A Slack `Connector` and a Slack `MessagingChannel` both exist — two artifacts for one product. Mitigated by the strict layering (mechanism vs semantic) and the shared transport, so there is no duplicated Slack-auth code. +- The `connector_action`-direct path can post to Slack *without* preference/inbox semantics. That is intended (it is the "raw API call" escape hatch), but docs must steer "notify a human" toward `notify`, not `connector_action`, so authors don't accidentally bypass mute/quiet-hours. + +--- + +## Status of the surrounding ADRs (so scope is clear) + +- **Built**: connector mechanism — `connector_action` baseline node, `engine.registerConnector()`, `@objectstack/connector-rest` (ADR-0018 addendum, merged 2026-05-31). +- **Draft, unbuilt**: `service-messaging`, the `notify` transport, the preference matrix, the inbox, `plugin-messaging-slack` (ADR-0012/0013). The `notify` node is still a no-op. +- **This ADR changes no code.** It records the layering decision and the routing rule so that when ADR-0012/0013 are implemented, the Slack channel is built *on* the connector substrate rather than beside it. + +## Follow-ups (not in this ADR) + +- Add a back-reference note to [ADR-0012 §2](./0012-notification-platform.md) and [ADR-0013 §9](./0013-bidirectional-messaging.md): "channel transport delegates to a `Connector` per ADR-0022." +- A `connector-slack` plugin (open, static-token) — would validate the registry with a second connector and immediately enable the `connector_action`-direct path; lands independently of the messaging stack. +- Revisit whether `plugin-email`'s transports should re-express as connectors once a second transport-heavy channel exists. diff --git a/packages/connectors/connector-slack/LICENSE b/packages/connectors/connector-slack/LICENSE new file mode 100644 index 0000000000..93fba8d887 --- /dev/null +++ b/packages/connectors/connector-slack/LICENSE @@ -0,0 +1,93 @@ +License text copyright (c) 2020 MariaDB Corporation Ab, All Rights Reserved. +"Business Source License" is a trademark of MariaDB Corporation Ab. + +Parameters + +Licensor: ObjectStack AI LLC +Licensed Work: ObjectStack Runtime: the BSL-licensed packages + of the ObjectStack monorepo as listed in LICENSING.md. + Copyright (c) 2026 ObjectStack AI LLC. +Additional Use Grant: You may make production use of the Licensed Work, provided + Your use does not include offering the Licensed Work to third + parties on a hosted or embedded basis in order to compete with + ObjectStack AI LLC's paid version(s) of the Licensed Work. For purposes + of this license: + + A "competitive offering" is a Product that is offered to third + parties on a paid basis, including through paid support + arrangements, that significantly overlaps with the capabilities + of ObjectStack AI LLC's paid version(s) of the Licensed Work. If Your + Product is not a competitive offering when You first make it + generally available, it will not become a competitive offering + later due to ObjectStack AI LLC releasing a new version of the Licensed + Work with additional capabilities. In addition, Products that + are not provided on a paid basis are not competitive. + + "Product" means software that is offered to end users to manage + in their own environments or offered as a service on a hosted + basis. + + "Embedded" means including the source code or executable code + from the Licensed Work in a competitive offering. "Embedded" + also means packaging the competitive offering in such a way + that the Licensed Work must be accessed or downloaded for the + competitive offering to operate. + + Hosting or using the Licensed Work(s) for internal purposes + within an organization is not considered a competitive + offering. ObjectStack AI LLC considers your organization to include all + of your affiliates under common control. + + For binding interpretive guidance on using ObjectStack AI LLC products + under the Business Source License, please visit our FAQ. + (see LICENSING.md in this repository) +Change Date: Four years from the date the Licensed Work is published. +Change License: Apache License, Version 2.0 + +For information about alternative licensing arrangements for the Licensed Work, +please contact licensing@objectstack.dev. + +Notice + +Business Source License 1.1 + +Terms + +The Licensor hereby grants you the right to copy, modify, create derivative +works, redistribute, and make non-production use of the Licensed Work. The +Licensor may make an Additional Use Grant, above, permitting limited production use. + +Effective on the Change Date, or the fourth anniversary of the first publicly +available distribution of a specific version of the Licensed Work under this +License, whichever comes first, the Licensor hereby grants you rights under +the terms of the Change License, and the rights granted in the paragraph +above terminate. + +If your use of the Licensed Work does not comply with the requirements +currently in effect as described in this License, you must purchase a +commercial license from the Licensor, its affiliated entities, or authorized +resellers, or you must refrain from using the Licensed Work. + +All copies of the original and modified Licensed Work, and derivative works +of the Licensed Work, are subject to this License. This License applies +separately for each version of the Licensed Work and the Change Date may vary +for each version of the Licensed Work released by Licensor. + +You must conspicuously display this License on each original or modified copy +of the Licensed Work. If you receive the Licensed Work in original or +modified form from a third party, the terms and conditions set forth in this +License apply to your use of that work. + +Any use of the Licensed Work in violation of this License will automatically +terminate your rights under this License for the current and all other +versions of the Licensed Work. + +This License does not grant you any right in any trademark or logo of +Licensor or its affiliates (provided that you may use a trademark or logo of +Licensor as expressly required by this License). + +TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON +AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS, +EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND +TITLE. diff --git a/packages/connectors/connector-slack/package.json b/packages/connectors/connector-slack/package.json new file mode 100644 index 0000000000..1cacbdc6a2 --- /dev/null +++ b/packages/connectors/connector-slack/package.json @@ -0,0 +1,36 @@ +{ + "name": "@objectstack/connector-slack", + "version": "7.3.0", + "license": "Apache-2.0", + "description": "Slack Web API connector for ObjectStack — registers `chat.postMessage` / `chat.update` / `call` actions on the automation engine's connector registry (ADR-0018 §Addendum, ADR-0022).", + "main": "dist/index.js", + "types": "dist/index.d.ts", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.mjs", + "require": "./dist/index.js" + } + }, + "scripts": { + "build": "tsup --config ../../../tsup.config.ts", + "test": "vitest run --passWithNoTests" + }, + "dependencies": { + "@objectstack/core": "workspace:*", + "@objectstack/spec": "workspace:*" + }, + "devDependencies": { + "@objectstack/service-automation": "workspace:*", + "@types/node": "^25.9.1", + "typescript": "^6.0.3", + "vitest": "^4.1.7" + }, + "keywords": [ + "objectstack", + "connector", + "slack", + "integration", + "messaging" + ] +} diff --git a/packages/connectors/connector-slack/src/connector-slack-plugin.test.ts b/packages/connectors/connector-slack/src/connector-slack-plugin.test.ts new file mode 100644 index 0000000000..74593f34fb --- /dev/null +++ b/packages/connectors/connector-slack/src/connector-slack-plugin.test.ts @@ -0,0 +1,78 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect } from 'vitest'; +import { LiteKernel } from '@objectstack/core'; +import { AutomationServicePlugin, type AutomationEngine } from '@objectstack/service-automation'; +import { ConnectorSlackPlugin } from './connector-slack-plugin.js'; + +/** A fetch stub mimicking the Slack Web API (HTTP 200 + ok in the body). */ +function stubSlack() { + const calls: Array<{ url: string; init: RequestInit }> = []; + const impl = (async (url: string, init: RequestInit) => { + calls.push({ url, init }); + return { + status: 200, + ok: true, + headers: { get: () => 'application/json' }, + json: async () => ({ ok: true, ts: '1700000000.000200', channel: 'C42' }), + text: async () => '{"ok":true}', + }; + }) as unknown as typeof fetch; + return { impl, calls }; +} + +describe('ConnectorSlackPlugin — end to end with the automation engine', () => { + it('registers the Slack connector so a connector_action flow can post a message', async () => { + const { impl, calls } = stubSlack(); + + const kernel = new LiteKernel(); + kernel.use(new AutomationServicePlugin()); + kernel.use(new ConnectorSlackPlugin({ token: 'xoxb-secret-token', fetchImpl: impl })); + await kernel.bootstrap(); + + const engine = kernel.getService('automation'); + + // The baseline dispatch node and the plugin-contributed connector are both present. + expect(engine.getRegisteredNodeTypes()).toContain('connector_action'); + expect(engine.getRegisteredConnectors()).toContain('slack'); + + engine.registerFlow('notify_channel', { + name: 'notify_channel', + label: 'Post to Slack', + type: 'autolaunched', + variables: [{ name: 'post.ok', type: 'boolean', isOutput: true }], + nodes: [ + { id: 'start', type: 'start', label: 'Start' }, + { + id: 'post', + type: 'connector_action', + label: 'Post to #sales', + connectorConfig: { + connectorId: 'slack', + actionId: 'chat.postMessage', + input: { channel: 'C42', text: 'Deal closed 🎉' }, + }, + }, + { id: 'end', type: 'end', label: 'End' }, + ], + edges: [ + { id: 'e1', source: 'start', target: 'post' }, + { id: 'e2', source: 'post', target: 'end' }, + ], + }); + + const result = await engine.execute('notify_channel'); + + expect(result.success).toBe(true); + // The Slack connector handled the dispatch: one POST with bearer auth + JSON body. + expect(calls).toHaveLength(1); + expect(calls[0].url).toBe('https://slack.com/api/chat.postMessage'); + expect(calls[0].init.method).toBe('POST'); + expect((calls[0].init.headers as Record).Authorization).toBe('Bearer xoxb-secret-token'); + expect(calls[0].init.body).toBe('{"channel":"C42","text":"Deal closed 🎉"}'); + // Slack's logical `ok` propagated back into the flow output. + expect(result.output).toEqual({ 'post.ok': true }); + + await kernel.shutdown(); + }); +}); diff --git a/packages/connectors/connector-slack/src/connector-slack-plugin.ts b/packages/connectors/connector-slack/src/connector-slack-plugin.ts new file mode 100644 index 0000000000..4728d378f8 --- /dev/null +++ b/packages/connectors/connector-slack/src/connector-slack-plugin.ts @@ -0,0 +1,79 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import type { Plugin, PluginContext } from '@objectstack/core'; +import type { Connector } from '@objectstack/spec/integration'; +import { createSlackConnector, type SlackConnectorOptions } from './slack-connector.js'; + +/** + * Minimal surface of the automation engine this plugin depends on — the + * connector registry from ADR-0018 §Addendum. Kept structural so the plugin + * needs no runtime dependency on `@objectstack/service-automation`. + */ +export interface ConnectorRegistrySurface { + registerConnector( + def: Connector, + handlers: Record< + string, + (input: Record, ctx: unknown) => Promise> + >, + ): void; + unregisterConnector(name: string): void; +} + +export interface ConnectorSlackPluginOptions extends SlackConnectorOptions {} + +/** + * ConnectorSlackPlugin — registers a Slack Web API connector on the automation + * engine. The second reference concrete connector (ADR-0018 §Addendum); it + * enables the ADR-0022 "raw API call" path — a flow's `connector_action` step + * can dispatch to `slack.chat.postMessage` without the messaging stack. + * + * If no automation engine is present the plugin logs and skips — the connector + * has nowhere to register, which is not an error. + */ +export class ConnectorSlackPlugin implements Plugin { + name = 'com.objectstack.connector.slack'; + version = '1.0.0'; + type = 'standard' as const; + // Ensure the automation engine (and its connector registry) is started first. + dependencies = ['com.objectstack.service-automation']; + + private readonly options: ConnectorSlackPluginOptions; + private connectorName?: string; + private automation?: ConnectorRegistrySurface; + + constructor(options: ConnectorSlackPluginOptions) { + this.options = options; + } + + async init(_ctx: PluginContext): Promise { + // No services to register; the connector is registered in start() once + // the automation engine is available. + } + + async start(ctx: PluginContext): Promise { + let automation: ConnectorRegistrySurface | undefined; + try { + automation = ctx.getService('automation'); + } catch { + automation = undefined; + } + + if (!automation || typeof automation.registerConnector !== 'function') { + ctx.logger.info('ConnectorSlackPlugin: no automation engine — Slack connector not registered'); + return; + } + + const { def, handlers } = createSlackConnector(this.options); + automation.registerConnector(def, handlers); + this.automation = automation; + this.connectorName = def.name; + ctx.logger.info(`ConnectorSlackPlugin: Slack connector '${def.name}' registered`); + } + + async stop(_ctx: PluginContext): Promise { + if (this.automation && this.connectorName) { + try { this.automation.unregisterConnector(this.connectorName); } catch { /* ignore */ } + } + } +} diff --git a/packages/connectors/connector-slack/src/index.ts b/packages/connectors/connector-slack/src/index.ts new file mode 100644 index 0000000000..39e956e810 --- /dev/null +++ b/packages/connectors/connector-slack/src/index.ts @@ -0,0 +1,28 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * @objectstack/connector-slack + * + * Slack Web API connector — a concrete connector (ADR-0018 §Addendum) and the + * second reference implementation after `@objectstack/connector-rest`. The + * baseline automation engine ships the `connector_action` dispatch node + an + * empty connector registry; this plugin populates it with a `slack` connector + * exposing `chat.postMessage`, `chat.update`, and a generic `call` action. + * + * This is the integration mechanism (ADR-0022 "raw API call" path), not the + * human-notification layer. Static bot-token auth only; OAuth2 install/refresh, + * credential vaulting, and multi-tenant lifecycle are the enterprise tier. + */ + +export { + createSlackConnector, + type SlackConnectorOptions, + type SlackConnectorBundle, + type SlackPostMessageInput, + type SlackResult, +} from './slack-connector.js'; +export { + ConnectorSlackPlugin, + type ConnectorSlackPluginOptions, + type ConnectorRegistrySurface, +} from './connector-slack-plugin.js'; diff --git a/packages/connectors/connector-slack/src/slack-connector.test.ts b/packages/connectors/connector-slack/src/slack-connector.test.ts new file mode 100644 index 0000000000..40b6abfc85 --- /dev/null +++ b/packages/connectors/connector-slack/src/slack-connector.test.ts @@ -0,0 +1,132 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, it, expect } from 'vitest'; +import { createSlackConnector } from './slack-connector.js'; + +// ─── Helpers ───────────────────────────────────────────────────────── + +interface CapturedCall { + url: string; + init: RequestInit; +} + +/** + * A fetch stub mimicking the Slack Web API: always HTTP 200, with the logical + * outcome carried in the JSON body's `ok` field. + */ +function stubSlack(responseBody: Record = { ok: true }) { + const calls: CapturedCall[] = []; + const impl = (async (url: string, init: RequestInit) => { + calls.push({ url, init }); + return { + status: 200, + ok: true, + headers: { get: () => 'application/json' }, + json: async () => responseBody, + text: async () => JSON.stringify(responseBody), + }; + }) as unknown as typeof fetch; + return { impl, calls }; +} + +function headersOf(call: CapturedCall): Record { + return (call.init.headers ?? {}) as Record; +} + +// ─── definition ────────────────────────────────────────────────────── + +describe('createSlackConnector — definition', () => { + it('declares a slack connector with the expected actions and bearer auth', () => { + const { def, handlers } = createSlackConnector({ token: 'xoxb-1' }); + + expect(def.name).toBe('slack'); + expect(def.type).toBe('api'); + expect(def.authentication).toEqual({ type: 'bearer', token: 'xoxb-1' }); + + const keys = (def.actions ?? []).map((a) => a.key); + expect(keys).toEqual(['chat.postMessage', 'chat.update', 'call']); + // Every declared action has a handler (the registry enforces this too). + for (const k of keys) expect(typeof handlers[k]).toBe('function'); + }); +}); + +// ─── chat.postMessage ──────────────────────────────────────────────── + +describe('createSlackConnector — chat.postMessage', () => { + it('POSTs JSON to the method URL with a bearer token and returns ok from the payload', async () => { + const { impl, calls } = stubSlack({ ok: true, ts: '1700000000.000100', channel: 'C123' }); + const { handlers } = createSlackConnector({ token: 'xoxb-secret', fetchImpl: impl }); + + const out = await handlers['chat.postMessage']( + { channel: 'C123', text: 'hello', thread_ts: '1699999999.0001' }, + {}, + ); + + expect(calls).toHaveLength(1); + expect(calls[0].url).toBe('https://slack.com/api/chat.postMessage'); + expect(calls[0].init.method).toBe('POST'); + expect(headersOf(calls[0]).Authorization).toBe('Bearer xoxb-secret'); + expect(calls[0].init.body).toBe('{"channel":"C123","text":"hello","thread_ts":"1699999999.0001"}'); + + expect(out.ok).toBe(true); + expect(out.status).toBe(200); + expect(out.body).toEqual({ ok: true, ts: '1700000000.000100', channel: 'C123' }); + expect(out.error).toBeUndefined(); + }); + + it('surfaces a Slack logical failure (HTTP 200, ok:false) without throwing', async () => { + const { impl } = stubSlack({ ok: false, error: 'channel_not_found' }); + const { handlers } = createSlackConnector({ token: 'xoxb-1', fetchImpl: impl }); + + const out = await handlers['chat.postMessage']({ channel: 'nope', text: 'hi' }, {}); + + expect(out.ok).toBe(false); + expect(out.error).toBe('channel_not_found'); + expect(out.status).toBe(200); + }); +}); + +// ─── chat.update + call ─────────────────────────────────────────────── + +describe('createSlackConnector — chat.update & generic call', () => { + it('chat.update hits the chat.update method', async () => { + const { impl, calls } = stubSlack({ ok: true }); + const { handlers } = createSlackConnector({ token: 'xoxb-1', fetchImpl: impl }); + + await handlers['chat.update']({ channel: 'C1', ts: '170.1', text: 'edited' }, {}); + expect(calls[0].url).toBe('https://slack.com/api/chat.update'); + expect(calls[0].init.body).toBe('{"channel":"C1","ts":"170.1","text":"edited"}'); + }); + + it('call dispatches to an arbitrary Web API method with params', async () => { + const { impl, calls } = stubSlack({ ok: true, channels: [] }); + const { handlers } = createSlackConnector({ token: 'xoxb-1', fetchImpl: impl }); + + const out = await handlers.call({ method: 'conversations.list', params: { limit: 50 } }, {}); + expect(calls[0].url).toBe('https://slack.com/api/conversations.list'); + expect(calls[0].init.body).toBe('{"limit":50}'); + expect(out.ok).toBe(true); + }); + + it('call throws when method is missing', async () => { + const { impl } = stubSlack(); + const { handlers } = createSlackConnector({ token: 'xoxb-1', fetchImpl: impl }); + await expect(handlers.call({ params: {} }, {})).rejects.toThrow(/method.*required/); + }); + + it('honours a custom baseUrl and merges defaultHeaders', async () => { + const { impl, calls } = stubSlack(); + const { handlers } = createSlackConnector({ + token: 'xoxb-1', + baseUrl: 'https://slack.example.test/api/', + defaultHeaders: { 'X-Trace': 'on' }, + fetchImpl: impl, + }); + + await handlers['chat.postMessage']({ channel: 'C1', text: 'x' }, {}); + expect(calls[0].url).toBe('https://slack.example.test/api/chat.postMessage'); + expect(headersOf(calls[0])['X-Trace']).toBe('on'); + // Auth still wins over defaultHeaders. + expect(headersOf(calls[0]).Authorization).toBe('Bearer xoxb-1'); + }); +}); diff --git a/packages/connectors/connector-slack/src/slack-connector.ts b/packages/connectors/connector-slack/src/slack-connector.ts new file mode 100644 index 0000000000..7298cec462 --- /dev/null +++ b/packages/connectors/connector-slack/src/slack-connector.ts @@ -0,0 +1,210 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +import type { Connector } from '@objectstack/spec/integration'; + +/** + * Slack connector — a *concrete* connector (ADR-0018 §Addendum) and the second + * reference implementation after `@objectstack/connector-rest`. It produces a + * {@link Connector} definition plus handlers for a small set of Slack Web API + * actions, which the baseline `connector_action` node dispatches to. + * + * Scope (ADR-0022 "raw API call" path): this is the *integration mechanism* + * for talking to Slack's API — "post this exact text to this channel". It is + * deliberately **not** the human-notification layer: there is no preference + * matrix, inbox, outbox, or thread/session semantics here. Those belong to a + * `MessagingChannel` (ADR-0012/0013), which may itself delegate its transport + * to this connector. + * + * Open-source scope: **static** auth only — a Slack **bot token** (`xoxb-…`), + * supplied by the caller and sent as a bearer credential. OAuth2 install/refresh, + * credential vaulting, and multi-tenant connection lifecycle are the enterprise + * tier (see `../cloud/docs/design/connector-tiering.md`) and are out of scope. + * + * Slack quirk: the Web API returns HTTP `200` even on logical failure, with the + * real outcome in the JSON body's `ok` field (and `error` on failure). Handlers + * therefore surface `ok` from the payload, not from the HTTP status, and never + * throw on a logical failure — the flow author branches on `${node.ok}`. + */ + +export interface SlackConnectorOptions { + /** Connector machine name (snake_case). Defaults to `slack`. */ + name?: string; + /** Human-readable label. Defaults to `Slack`. */ + label?: string; + /** Slack bot token (`xoxb-…`), sent as `Authorization: Bearer `. */ + token: string; + /** Web API base URL. Defaults to `https://slack.com/api`. */ + baseUrl?: string; + /** Headers merged into every request (request-level headers win). */ + defaultHeaders?: Record; + /** Injected for tests; defaults to the global `fetch`. */ + fetchImpl?: typeof fetch; +} + +/** Input accepted by `chat.postMessage`. */ +export interface SlackPostMessageInput { + /** Channel id (`C…`), user id (`U…`), or channel name (`#general`). */ + channel: string; + /** Message text (fallback when `blocks` are present). */ + text?: string; + /** Thread root `ts` to reply into an existing thread. */ + thread_ts?: string; + /** Block Kit blocks. */ + blocks?: unknown[]; + [key: string]: unknown; +} + +/** Generic Slack Web API result envelope. */ +export interface SlackResult { + /** Slack's logical success flag (from the JSON body, not the HTTP status). */ + ok: boolean; + /** HTTP status (almost always 200 for the Slack Web API). */ + status: number; + /** Full parsed Slack payload. */ + body: Record; + /** Slack error code when `ok` is false (e.g. `channel_not_found`). */ + error?: string; +} + +/** A connector definition paired with its action handlers, ready for registerConnector(). */ +export interface SlackConnectorBundle { + def: Connector; + handlers: Record< + string, + (input: Record, ctx: unknown) => Promise> + >; +} + +export function createSlackConnector(opts: SlackConnectorOptions): SlackConnectorBundle { + const name = opts.name ?? 'slack'; + const baseUrl = (opts.baseUrl ?? 'https://slack.com/api').replace(/\/+$/, ''); + const doFetch = opts.fetchImpl ?? fetch; + + const def: Connector = { + name, + label: opts.label ?? 'Slack', + type: 'api', + description: 'Slack Web API connector (static bot-token auth). Post and update messages, or call any Web API method.', + icon: 'slack', + authentication: { type: 'bearer', token: opts.token }, + // Defaulted by ConnectorSchema; set explicitly so the literal satisfies + // the (post-parse) Connector output type. + status: 'active', + enabled: true, + connectionTimeoutMs: 30000, + requestTimeoutMs: 30000, + actions: [ + { + key: 'chat.postMessage', + label: 'Post Message', + description: 'Post a message to a channel, DM, or thread (Slack chat.postMessage).', + inputSchema: { + type: 'object', + required: ['channel'], + properties: { + channel: { type: 'string', description: 'Channel id, user id, or #name' }, + text: { type: 'string', description: 'Message text' }, + thread_ts: { type: 'string', description: 'Thread root ts to reply into' }, + blocks: { type: 'array', description: 'Block Kit blocks' }, + }, + }, + outputSchema: slackOutputSchema(), + }, + { + key: 'chat.update', + label: 'Update Message', + description: 'Edit an existing message (Slack chat.update).', + inputSchema: { + type: 'object', + required: ['channel', 'ts'], + properties: { + channel: { type: 'string', description: 'Channel id' }, + ts: { type: 'string', description: 'Timestamp of the message to update' }, + text: { type: 'string', description: 'New message text' }, + blocks: { type: 'array', description: 'New Block Kit blocks' }, + }, + }, + outputSchema: slackOutputSchema(), + }, + { + key: 'call', + label: 'Call Web API Method', + description: 'Escape hatch — call any Slack Web API method with arbitrary params.', + inputSchema: { + type: 'object', + required: ['method'], + properties: { + method: { type: 'string', description: 'Web API method, e.g. conversations.list' }, + params: { type: 'object', description: 'Method parameters (JSON body)' }, + }, + }, + outputSchema: slackOutputSchema(), + }, + ], + }; + + /** POST a JSON body to a Slack Web API method and normalise the result. */ + async function callSlack(method: string, params: Record): Promise { + const headers: Record = { + 'Content-Type': 'application/json; charset=utf-8', + ...opts.defaultHeaders, + Authorization: `Bearer ${opts.token}`, + }; + + const response = await doFetch(`${baseUrl}/${method}`, { + method: 'POST', + headers, + body: JSON.stringify(params), + }); + + // The Slack Web API always answers with JSON; `ok` is the real outcome. + const body = (await response.json()) as Record; + const ok = body.ok === true; + return { + ok, + status: response.status, + body, + error: ok ? undefined : (body.error as string | undefined), + }; + } + + async function postMessage(input: Record): Promise> { + return toRecord(await callSlack('chat.postMessage', input)); + } + + async function update(input: Record): Promise> { + return toRecord(await callSlack('chat.update', input)); + } + + async function call(input: Record): Promise> { + const method = String(input.method ?? ''); + if (!method) throw new Error("slack 'call' action: 'method' is required"); + const params = (input.params as Record) ?? {}; + return toRecord(await callSlack(method, params)); + } + + return { + def, + handlers: { + 'chat.postMessage': postMessage, + 'chat.update': update, + call, + }, + }; +} + +function toRecord(r: SlackResult): Record { + return { ok: r.ok, status: r.status, body: r.body, error: r.error }; +} + +function slackOutputSchema(): Record { + return { + type: 'object', + properties: { + ok: { type: 'boolean', description: "Slack's logical success flag" }, + status: { type: 'number', description: 'HTTP status' }, + body: { type: 'object', description: 'Full Slack response payload' }, + error: { type: 'string', description: 'Slack error code when ok is false' }, + }, + }; +} diff --git a/packages/connectors/connector-slack/tsconfig.json b/packages/connectors/connector-slack/tsconfig.json new file mode 100644 index 0000000000..385be7ea89 --- /dev/null +++ b/packages/connectors/connector-slack/tsconfig.json @@ -0,0 +1,10 @@ +{ + "extends": "../../../tsconfig.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src", + "types": ["node"] + }, + "include": ["src/**/*"], + "exclude": ["dist", "node_modules", "**/*.test.ts"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 03c434ec8d..8aa2ce5de7 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -684,6 +684,28 @@ importers: specifier: ^4.1.7 version: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/coverage-v8@4.1.7)(happy-dom@20.9.0)(msw@2.14.6(@types/node@25.9.1)(typescript@6.0.3))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)) + packages/connectors/connector-slack: + dependencies: + '@objectstack/core': + specifier: workspace:* + version: link:../../core + '@objectstack/spec': + specifier: workspace:* + version: link:../../spec + devDependencies: + '@objectstack/service-automation': + specifier: workspace:* + version: link:../../services/service-automation + '@types/node': + specifier: ^25.9.1 + version: 25.9.1 + typescript: + specifier: ^6.0.3 + version: 6.0.3 + vitest: + specifier: ^4.1.7 + version: 4.1.7(@opentelemetry/api@1.9.1)(@types/node@25.9.1)(@vitest/coverage-v8@4.1.7)(happy-dom@20.9.0)(msw@2.14.6(@types/node@25.9.1)(typescript@6.0.3))(vite@8.0.14(@types/node@25.9.1)(esbuild@0.28.0)(jiti@2.7.0)(tsx@4.22.3)(yaml@2.9.0)) + packages/console: {} packages/core: From 413c5928ccf678c19061953407b286342bd3c71f Mon Sep 17 00:00:00 2001 From: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Date: Sun, 31 May 2026 23:59:10 +0800 Subject: [PATCH 3/3] docs(adr-0021): resolve D-A/B/C and Q1-Q3 on the AI-author / human-review criterion - D-A: dataset is the ONE author-facing semantic type; compiles to the existing Cube runtime; retire/absorb the author-facing Cube DSL. - D-B: take high-prior names dataset/measure/dimension; rename seed Dataset -> Seed; Cube Dimension/Metric absorbed (collision dissolves). - D-C: reuse Cube NativeSQLStrategy join path but make RLS + per-joined- object tenant scoping mandatory; v1 joins only along declared relations. - Q1: derived measures first-class now, CLOSED (reference measures by name only; no raw fields/SQL). - Q2: deferred; runtimeFilter only in v1. - Q3: deferred; conformed dimensions in a follow-up ADR. Rewrites D1 schema to a declarative surface smaller than QuerySchema (object + include relationships + dims/measures + derived), no raw SQL, joins derived from the object graph. Adds the six AI-author principles that drive the decisions. WS2 drops XL->L; estimate ~4-6 weeks. --- .../0021-analytics-dataset-semantic-layer.md | 114 ++++++++++++------ 1 file changed, 78 insertions(+), 36 deletions(-) diff --git a/docs/adr/0021-analytics-dataset-semantic-layer.md b/docs/adr/0021-analytics-dataset-semantic-layer.md index 697547ea8d..9d466bdc5d 100644 --- a/docs/adr/0021-analytics-dataset-semantic-layer.md +++ b/docs/adr/0021-analytics-dataset-semantic-layer.md @@ -1,6 +1,6 @@ # ADR-0021: Analytics — one semantic `dataset` layer, `report` / `dashboard` become pure presentation -**Status**: Proposed — **revised 2026-05-31 after an implementation scan** (see "Implementation scan" near the end; the original "the engine already does joins" premise was corrected) +**Status**: Proposed — **revised 2026-05-31**: implementation scan folded in, and the gating decisions **D-A / D-B / D-C resolved** + open questions Q1–Q3 closed, optimised for the AI-authors / human-reviews design center (see "Decision" and "Resolved decisions") **Deciders**: ObjectStack Protocol Architects **Builds on**: [ADR-0019](./0019-app-as-consumer-unit.md) + [ADR-0020](./0020-state-machine-converge-and-enforce.md) (the "one engine, fold the parasitic concept into its host" principle — applied here to analytics), [ADR-0017](./0017-object-has-many-view.md) (object-bound ListView is the row-level lens), [ADR-0010](./0010-nl-to-flow-authoring.md) + [ADR-0011](./0011-actions-as-ai-tools.md) (AI authoring is the design center) **Consumers**: `@objectstack/spec` (`ui/report.zod.ts`, `ui/dashboard.zod.ts`, `ui/view.zod.ts`, `data/query.zod.ts`, `kernel/metadata-type-schemas.ts`), `@objectstack/objectql` (query engine), `@objectstack/analytics-service`, all `examples/*` @@ -29,7 +29,7 @@ This produces three structural defects that are fatal for an **enterprise core-b 2. **Triple double-write ⇒ metric drift.** "Revenue" is defined three times in three different grammars. Finance numbers diverge across a report, a dashboard tile, and a list chart. This is a governance red line. 3. **No single source of truth ⇒ no drill-through, no certification, no reuse.** A dashboard tile cannot drill into the report behind it because there is no shared definition behind either. -**Decision.** Introduce **one semantic layer — `dataset`** — a named, reusable analytical query (a thin wrapper over `QuerySchema`) that declares its **dimensions** and **measures** *by name*. Then **collapse `report` and `dashboard` to pure presentation** that binds to a dataset by reference and selects dimensions/measures *by name* — never re-declaring `object` / `field` / `aggregate`. Chart visualization config (`ChartConfigSchema`) stays shared (it already was). `report` and `dashboard` remain **two metadata types** because their *render grammars* genuinely differ (pivot grid vs. widget canvas) — but neither owns a query anymore. +**Decision.** Introduce **one semantic layer — `dataset`** — a named, reusable analytical definition (base object + included relationships + declared **dimensions** and **measures**) that **compiles to the existing Cube analytics runtime**, *not* a wrapper exposing raw `QuerySchema` joins. Then **collapse `report` and `dashboard` to pure presentation** that binds to a dataset by reference and selects dimensions/measures *by name* — never re-declaring `object` / `field` / `aggregate`. Chart visualization config (`ChartConfigSchema`) stays shared (it already was). `report` and `dashboard` remain **two metadata types** because their *render grammars* genuinely differ (pivot grid vs. widget canvas) — but neither owns a query anymore. This is the industry-convergent shape (Looker LookML / Power BI dataset+model / dbt metrics / Salesforce CRM-Analytics dataset): **a governed semantic layer below; thin presentations above.** It keeps Airtable's authoring ergonomics (an inline single-object dataset auto-desugars, see §4) while gaining the governance Airtable lacks. @@ -53,13 +53,13 @@ Mature analytics stacks all converge on a three-layer model. We adopt it verbati ``` ┌─ Presentation report (pivot) · dashboard (canvas) · listView (row lens) │ │ reference by name, select dims/measures -├─ Semantic dataset — named query + declared dimensions + declared measures ← single source of truth +├─ Semantic dataset — base object + included relationships + declared dimensions/measures ← single source of truth │ │ compiles to -└─ Engine QuerySchema (joins / aggregations / groupBy / having / window) ← already exists +└─ Runtime Cube analytics runtime (IAnalyticsService) — RLS/tenant-enforced join + aggregation ``` -- **Engine** (`query.zod.ts`) — exists, untouched. It is the SQL/AST. -- **Semantic** (`dataset`, NEW) — names the query and, crucially, names its **dimensions** (groupable axes) and **measures** (aggregatable values, with format + certification). `revenue` is defined *once* here. +- **Runtime** — the **already-implemented Cube analytics runtime** (`IAnalyticsService`, `service-analytics`) executes joins + aggregations. The dataset *compiles to* it (resolved decision **D-A=(c)**). We do **not** build a third query path, and we do **not** expose the raw `QuerySchema` join grammar to authors. +- **Semantic** (`dataset`, NEW) — a **declarative, closed** surface that is deliberately *smaller* than `QuerySchema`: an author declares a base object, which **relationships** to include (by name — joins are *derived from the object graph*, never hand-written), and the **dimensions** (groupable axes) and **measures** (aggregatable values, with format + certification). `revenue` is defined *once* here. No raw SQL, no hand-authored join predicates. - **Presentation** — `report` / `dashboard` / `listView`-chart reference a dataset and pick dimensions/measures *by name*. Zero query fields. ### Why widgets bind to `dataset`, not to `report` @@ -107,7 +107,20 @@ Airtable's flat model is exactly our current `DashboardWidget`. It suits Airtabl ### Design center: AI authors this -Per ADR-0010/0011 the author is increasingly an AI. A named `dataset` with declared `measures: [{ name: "revenue", aggregate: "sum", field: "amount" }]` gives the model a **stable vocabulary**: a widget says `measures: ["revenue"]` and the model cannot invent a divergent `valueField/aggregate` pair. The semantic layer is also an **anti-hallucination guardrail** — the legal dimensions/measures are enumerable, so an Agent picks from a closed set instead of guessing field names. This mirrors the ADR-0020 reasoning ("meet the model where its priors are; make the legal set introspectable"). +Per ADR-0010/0011 the author is increasingly an AI and the human is a reviewer. This is **not a soft constraint — it is the criterion that resolves every open decision below.** Its imperative is singular: **shrink "what an author may write" to the smallest, most closed, most declarative surface possible.** A named `dataset` with declared `measures: [{ name: "revenue", aggregate: "sum", field: "amount" }]` gives the model a **stable, enumerable vocabulary**: a widget says `measures: ["revenue"]` and cannot invent a divergent `valueField/aggregate` pair. + +Six principles follow, and they drive the resolved decisions: + +| Principle | Why (AI-writes / human-reviews) | +|---|---| +| **One legal way to express a thing** | Multiple shapes are a hallucination trap (ADR-0020). The model picks the wrong one of N. | +| **Closed, enumerable, introspectable sets** | The AI selects from declared measures/dimensions instead of inventing fields — the strongest guardrail. An Agent can ask "what measures exist here?" exactly like ADR-0020's "what state transitions are legal?" | +| **Zero raw SQL / zero raw expressions** | Every escape hatch is at once a hallucination source, an injection risk, and an un-reviewable blob. Reviewing `measures: ["revenue"]` is O(1); auditing a hand-written join+SQL is O(n). | +| **Joins derived from the relationship graph, never hand-authored** | The object model already declares `lookup`/`master_detail`. The AI says "include `account.region`"; the compiler derives the join. The AI never writes an `ON` clause — the single biggest safety win. | +| **Safety enforced by the engine, not the author** | The AI will not reliably add a tenant/RLS filter and a reviewer will not reliably catch a missing one. RLS must be automatic (drives D-C). | +| **`certified` measures are the review checkpoint** | A human blesses a measure once; the AI reuses it; reviewing AI output collapses to "did it use certified measures correctly," not "is this query correct." | + +This is the **third application of the ADR-0019/0020 "one engine, converge the shapes" principle**: analytics intent is today expressible four ways (Cube DSL, `QuerySchema`, inline report query, inline widget query). The AI-author center *demands* converging to **one** author-facing semantic shape. --- @@ -117,14 +130,24 @@ Three decisions. ### D1 — Introduce `dataset` as the single analytical source of truth -A new top-level metadata type `dataset`. It is a **thin wrapper over `QuerySchema`** plus a declared semantic contract: +A new top-level metadata type `dataset`. It is **not** a wrapper over `QuerySchema` — that would expose hand-authored joins, exactly the surface the AI-author center forbids. Instead it is a **declarative, closed surface, deliberately smaller than `QuerySchema`**, that **compiles to the Cube analytics runtime** (D-A=(c)). The author declares a base object, which **relationships to include** (joins derived from the object graph), and named dimensions/measures. No raw SQL, no `ON` clauses, no window/having grammar in the author surface. ```ts // packages/spec/src/ui/dataset.zod.ts (NEW) +// +// Naming (D-B): this type takes the high-prior names `dataset` / `measure` / +// `dimension` (LookML/dbt/Cube/PowerBI vocabulary — densest in the model's +// priors). The colliding seed `DatasetSchema` is renamed to `Seed`/`SeedData` +// (a more accurate name anyway); the Cube layer's `Dimension`/`Metric` are +// ABSORBED here (D-A converges them — they are the same concept), so the +// collision dissolves rather than being worked around. + export const DimensionSchema = z.object({ name: SnakeCaseIdentifierSchema, // referenced by presentations label: I18nLabelSchema.optional(), - field: z.string(), // resolves within the dataset query (joins included) + /** A field on the base object, OR a `relationship.field` path. The join is + * DERIVED from the declared relationship — the author never writes a predicate. */ + field: z.string(), type: z.enum(['string','number','date','boolean','lookup']).optional(), dateGranularity: DateGranularity.optional(),// default bucketing for date dims }); @@ -133,10 +156,18 @@ export const MeasureSchema = z.object({ name: SnakeCaseIdentifierSchema, // e.g. "revenue" — defined ONCE label: I18nLabelSchema.optional(), aggregate: AggregationFunction, // reuse query.zod enum (sum/avg/count/...) - field: z.string().optional(), // optional for count(*) + field: z.string().optional(), // base/relationship field; optional for count(*) filter: FilterConditionSchema.optional(), // measure-scoped filter (e.g. won_amount) format: z.string().optional(), // "$0,0.00", "0.0%" - certified: z.boolean().default(false), // governance: blessed metric + certified: z.boolean().default(false), // governance: blessed metric (the review checkpoint) + + /** Q1 RESOLVED — derived measures are first-class from day one, but CLOSED: + * a derived measure references OTHER measures BY NAME only. No raw fields, + * no raw SQL. `{ ratio: 'won_amount', of: 'total_amount' }` → won/total. */ + derived: z.object({ + op: z.enum(['ratio','sum','difference','product']), + of: z.array(SnakeCaseIdentifierSchema), // names of other measures in this dataset + }).optional(), }); export const DatasetSchema = z.object({ @@ -144,8 +175,16 @@ export const DatasetSchema = z.object({ label: I18nLabelSchema, description: I18nLabelSchema.optional(), - /** The FROM/JOIN/WHERE — the full engine grammar, reused verbatim. */ - query: QuerySchema, // joins, filter, having, window — all available + /** Base object — the FROM. */ + object: z.string(), + + /** Relationships to include, BY NAME (lookup/master_detail field names on the + * object graph). Joins are compiled from these — the author writes no ON clause. + * v1 (D-C): only declared relationships are joinable; no arbitrary predicates. */ + include: z.array(z.string()).optional(), + + /** Definition-level filter (the dataset's intrinsic scope, e.g. "non-deleted"). */ + filter: FilterConditionSchema.optional(), /** The semantic contract presentations bind to. */ dimensions: z.array(DimensionSchema), @@ -156,7 +195,7 @@ export const DatasetSchema = z.object({ }); ``` -Security/RLS is inherited from the objects the `query` touches — there is **one** place to reason about access, not three. +**RLS/tenant is enforced by the runtime, not declared here** (D-C). The dataset compiles to a Cube query whose execution applies the sharing middleware's read filter **per joined object** — there is **one** place to reason about access, and the author cannot forget it. ### D2 — `report` becomes a pure pivot presentation over a dataset @@ -238,7 +277,7 @@ Dashboard-level `globalFilters` and `dateRange` bind to **dimension names**, not ## Consequences **Gained** -- **Joins everywhere, for free.** Any report/widget can be multi-object because the dataset's `query` is the full engine. The #1 enterprise blocker is gone. +- **Joins everywhere, safely.** Any report/widget can be multi-object because the dataset includes relationships that compile to the Cube runtime's join path (with RLS enforced per joined object, D-C). The #1 enterprise blocker is gone — without exposing hand-authored joins. - **One definition of every metric.** `revenue` lives in one dataset measure; every surface references it. No drift; `certified` enables governance. - **Drill-through is free.** A widget and the report behind it share a dataset, so a tile can `drillTo` a report or expand to underlying records natively. - **Smaller protocol.** Three inline query grammars (`Report.columns/groupings`, `Widget.category/value/aggregate/measures`, `ListChartConfig`) delete down to one (`dataset.dimensions/measures`). Net schema shrinks. @@ -259,8 +298,8 @@ The objection to a semantic layer is "now a one-number KPI needs a whole dataset // authoring sugar — single object, no named dataset needed widget: { id: 'open_deals', viz: { type: 'metric' }, - dataset: { query: { object: 'opportunity', filter: { stage: { $ne: 'closed' } } }, - measures: [{ name: 'v', aggregate: 'count' }] }, + dataset: { object: 'opportunity', filter: { stage: { $ne: 'closed' } }, + dimensions: [], measures: [{ name: 'v', aggregate: 'count' }] }, measures: ['v'], } ``` @@ -289,7 +328,7 @@ A deterministic codemod runs over all package/app metadata. No parallel old+new | Old | New | |---|---| -| `Report{ objectName, columns, groupingsDown/Across, filter }` | extract `dataset{ query:{object,filter}, dimensions:(groupings), measures:(aggregate columns) }`; `Report{ dataset, rows, columns, values }` | +| `Report{ objectName, columns, groupingsDown/Across, filter }` | extract `dataset{ object, filter, dimensions:(groupings), measures:(aggregate columns) }`; `Report{ dataset, rows, columns, values }` | | `Report{ type:'joined', blocks }` | one dataset per block → `Report{ sections:[{dataset}] }` | | `DashboardWidget{ object, categoryField, valueField, aggregate, filter }` | anonymous dataset (inline) or named if shared; `widget{ dataset, dimensions:[category], measures:[{aggregate,valueField}→name] }` | | `DashboardWidget{ measures[] }` (multi-measure) | dataset `measures[]` + widget `measures:[names]` | @@ -333,28 +372,29 @@ The ADR's `DatasetSchema` / `DimensionSchema` / `MeasureSchema` are all taken: ` - **Registration touches 4 surfaces**, not 1: `kernel/metadata-type-schemas.ts`, `kernel/metadata-plugin.zod.ts` (enum + `DEFAULT_METADATA_TYPE_REGISTRY`, `loadOrder` < report/dashboard), `shared/metadata-collection.zod.ts` (`MAP_SUPPORTED_FIELDS` + `PLURAL_TO_SINGULAR`), `objectql/engine.ts` `metadataArrayKeys` (**two** lists, L806 + L961). - **Migration scope:** 7 reports (2 files) + 64 widgets (4 files; heaviest `chart-gallery.dashboard.ts` = 38) + 2 chart-views (2 files) = **8 source files, two inline shapes**. Tests to rewrite: `view.test.ts` (214) + `dashboard.test.ts` (146) + `report.test.ts` (51) + `report-service.test.ts` + `view-expand.test.ts` + 3 example integration tests. No JSON/seed instance data. JSON-schema regenerates via `pnpm gen:schema`. -### Pre-work decisions (gate everything) +### Resolved decisions (gate everything) — decided 2026-05-31 on the AI-author / human-review criterion -| # | Decision | Options | Recommendation | +| # | Decision | **Resolution** | Rationale (AI writes / human reviews) | |---|---|---|---| -| D-A | Relationship to the existing Cube layer | (a) new type wrapping `QuerySchema`, build joins into the engine; (b) dataset **is** Cube, renamed/extended; (c) dataset **compiles to** `AnalyticsQuery`, reuse Cube runtime | **(c) / (b)** — reuse the implemented, join-capable Cube runtime; don't build a third layer | -| D-B | Naming collisions | rename (`AnalyticsDataset`/`Measure`) or namespace | rename; literal ADR names can't land | -| D-C | Join execution + safety | build join/having into `IDataEngine`+SQL driver; **or** route via Cube `NativeSQLStrategy` **and add RLS + tenant scoping** to that path | reuse Cube path **+ harden RLS/tenant** (enterprise red line) | +| **D-A** | Relationship to the existing Cube layer | **(c) — `dataset` is the one author-facing semantic type; it compiles to the existing Cube runtime; the author-facing Cube DSL is retired/absorbed.** Don't build a third layer; don't keep two author surfaces. | Converging to one shape is the ADR-0019/0020 principle. The dataset surface is *smaller* than both Cube-with-raw-SQL and `QuerySchema` — the minimal closed surface the model can't misuse. | +| **D-B** | Naming collisions | **Take the high-prior names `dataset` / `measure` / `dimension` for the analytics type.** Rename the seed `Dataset` → `Seed`/`SeedData` (more accurate, rarely AI-authored). The Cube `Dimension`/`Metric` are absorbed by D-A, so that collision dissolves. `measure` over `metric` (majority vocabulary). | Give the best-prior name to the surface the AI authors most. Seed data is the lower-AI-traffic collider, so it yields the name. | +| **D-C** | Join execution + safety | **Reuse the Cube `NativeSQLStrategy` join path, but make RLS + per-joined-object tenant scoping mandatory and automatic; and in v1 allow joins ONLY along declared relationships (no arbitrary predicates).** | The AI won't reliably add tenant/RLS filters and a reviewer won't reliably catch a missing one — so the engine must enforce it. Relationship-only joins bound both the safety surface and the AI surface. | -### Task list (workstreams; S≈½d, M≈1–3d, L≈1wk, XL≈2wk+; ⛔ = blocked by D-A/B/C) +### Task list (workstreams; S≈½d, M≈1–3d, L≈1wk, XL≈2wk+) — updated for the resolved decisions -**WS0 · Design close-out (blocks all) — M** -- ⛔ Resolve D-A / D-B / D-C; re-merge this ADR with the chosen direction. +**WS0 · Design close-out — ✅ DONE** +- D-A / D-B / D-C resolved (above); this revision re-merges the ADR with the chosen direction (dataset → compiles to Cube; high-prior names + rename seed; relationship-only joins with engine-enforced RLS). **WS1 · Spec / Schema (this repo) — L** -- `ui/dataset.zod.ts` (resolved names; dimensions/measures; `query` *or* mapping to Cube). +- `ui/dataset.zod.ts` per §D1 (names per D-B; `object` + `include` relationships + dimensions/measures + derived measures; **no `QuerySchema` exposure**). Rename seed `data/dataset.zod.ts` `Dataset` → `Seed`/`SeedData` and update its consumers (`stack.zod.ts`, `seed-loader-service.ts`, `metadata-collection.zod.ts`). Absorb the Cube `Dimension`/`Metric` types (D-A). - Refactor `report.zod.ts` (→ `dataset`/`rows`/`columns`/`values`/`sections`), `dashboard.zod.ts` (widget → `dataset`/`report`/`dimensions`/`measures`/`viz`/`drillTo` + `superRefine`; globalFilters/dateRange → dimension names), `view.zod.ts` (drop `ListChartConfigSchema` + `type:'chart'`). - Wire the 4 registration surfaces (above). Rewrite `report.form.ts`/`dashboard.form.ts` + add `dataset.form.ts`. Adjust `analytics-service`/`report-service`/`ui-service` contracts. Run `pnpm gen:schema`. -**WS2 · Runtime / engine (this repo) — XL (bottleneck)** -- ⛔ Join execution (engine join/having, or route dataset → Cube `NativeSQLStrategy`). -- ⛔ Add RLS + tenant safety to the join path. -- Dataset executor (dataset + selected dims/measures + runtimeFilter/compareTo → executable query → chart data); server-side `compareTo` time-shift; dataset metadata loading. +**WS2 · Runtime / engine (this repo) — L (was XL; halved by D-A=(c))** +- **Dataset → Cube compiler**: lower a `dataset` (object + include + dimensions/measures + derived) to an `AnalyticsQuery`/Cube definition; resolve `include` relationship names to join specs from the object graph. +- **Harden the Cube `NativeSQLStrategy` join path**: inject the sharing-middleware read filter + tenant scope **per joined object** (D-C — the current raw-SQL path bypasses both); reject any join not backed by a declared relationship. +- Derived-measure evaluation (Q1): compute ratio/sum/difference/product over already-aggregated named measures. +- Dataset executor (dataset + selected dims/measures + `runtimeFilter`/`compareTo` → Cube query → chart data); server-side `compareTo` time-shift; dataset metadata loading. **WS3 · Frontend + designers (sibling repo `objectui`, cross-repo) — XL** - Rework `DashboardRenderer.getComponentSchema()` and `useReportData.ts` to bind to dataset+measures; adapt `data-objectstack` `aggregate()`; convert `WidgetConfigPanel`/`DashboardEditor`/`ReportDesigner`/`ReportConfigPanel` from field pickers to dataset+measure pickers; new dataset editor + picker; bump `.objectui-sha`. @@ -365,10 +405,12 @@ The ADR's `DatasetSchema` / `DimensionSchema` / `MeasureSchema` are all taken: ` **WS5 · Docs / skills (this repo) — M** - ~9 mdx pages + `meta.json` nav; `skills/objectstack-ui/SKILL.md` (major) + `objectstack-query` (minor). -**Critical path:** WS0 → (WS1 ∥ WS2) → WS3 → WS4. **Rough order of magnitude:** ~6–8 weeks for one senior engineer across both repos; choosing D-A = (c) reuse Cube roughly halves WS2. +**Critical path:** WS1 ∥ WS2 → WS3 → WS4 (WS0 done). **Rough order of magnitude:** ~4–6 weeks for one senior engineer across both repos — D-A=(c) reusing the Cube runtime turned WS2 from XL to L. + +## Resolved open questions (Q1–Q3) -## Open questions +Closed 2026-05-31 on the AI-author / human-review criterion — the throughline is *shrink the author surface; defer anything that adds a writable concept.* -1. **Calculated/derived measures** (e.g. `win_rate = won / total`) — introduce a `MeasureSchema.expression` (measure-over-measures) now, or defer to a follow-up ADR? Leaning: define the field now, implement later. -2. **Dataset parameters** (a dataset templated on a runtime value, e.g. `:region`) vs. pushing all scoping to `runtimeFilter`. Leaning: `runtimeFilter` only in v1; parameters later if needed. -3. **Cross-dataset dashboard filters** — a global filter spanning widgets backed by *different* datasets requires a shared dimension name convention. Leaning: match by dimension `name`; document the convention. +1. **Calculated/derived measures — RESOLVED: first-class now, but closed.** AI authoring will demand ratios/derived metrics immediately; with no slot it would fake fields or mis-author formulas. So `MeasureSchema.derived` ships in v1 (see §D1), constrained to **reference other measures by name only** — no raw fields, no raw SQL — keeping it enumerable and reviewable. Runtime lands with WS2. +2. **Dataset parameters — RESOLVED: deferred; `runtimeFilter` only in v1.** Parameters (`:region` templating) are an imperative concept and another hallucination surface; `runtimeFilter` covers global-filter/report-scope declaratively. Fewer writable concepts wins. Revisit only if a real need can't be expressed as a filter. +3. **Cross-dataset dashboard filters — RESOLVED: deferred; conformed dimensions later.** A global filter spanning widgets on *different* datasets should not rely on fragile dimension-`name` string matching. The long-term answer is **conformed dimensions** that resolve to the same underlying `object.field`, with the filter binding to that semantic key. v1 supports single-dataset dashboards; conformed dimensions are a follow-up ADR.