diff --git a/.changeset/ai-wildcard-to-zero.md b/.changeset/ai-wildcard-to-zero.md new file mode 100644 index 0000000000..8cfe6f751e --- /dev/null +++ b/.changeset/ai-wildcard-to-zero.md @@ -0,0 +1,31 @@ +--- +"@objectstack/runtime": patch +--- + +test(client,runtime): the last wildcard was wrong evidence, not weak — AI ratchet 3 → 0 (#3718) + +The capstone (#3642) ratcheted "matched only by a `**` family" as weaker +evidence, to be driven down by enumerating each dynamic family. 60 → 3 after +#3656. The last 3 were `ai.nlq` / `ai.suggest` / `ai.insights` on `* /ai/**`. + +Enumerating that family (in `cloud`, where `service-ai` lives) showed the +wildcard had not been weak evidence but **wrong** evidence. `buildAIRoutes()` +mounts 12 routes — `chat`, `chat/stream`, `complete`, `models`, `status`, +`effective-model`, six `conversations` — and **none** is `/nlq`, `/suggest` or +`/insights`. The SDK's entire AI namespace is dead, the entire real AI surface +is unexpressed by the SDK, and the two sets are disjoint (#3718). + +The old row's note even claimed the client "expresses nlq/suggest/insights +against the REST AI routes". That was never verified and is false: +`DEFAULT_AI_ROUTES` declares them but has no runtime consumer (only the spec's +own test reads it), and `aiNlq?`/`aiSuggest?`/`aiInsights?` are optional +protocol methods nothing implements. + +`/api/v1/ai/` becomes a bounded prefix exemption alongside the control plane — +two cross-repo surfaces, both ledgered in `cloud` — and the wildcard-only +assertion becomes `toBe(0)`, not a ratchet: every matched call now rests on an +exact enumerated route. Mutation-checked in both directions (removing the +exemption re-exposes exactly the 3, and the pre-change count was verified to be +exactly those 3 and nothing else). + +Test-and-comment changes only; no runtime behaviour is affected. diff --git a/content/docs/api/client-sdk.mdx b/content/docs/api/client-sdk.mdx index 6a2bac5989..97ebc79037 100644 --- a/content/docs/api/client-sdk.mdx +++ b/content/docs/api/client-sdk.mdx @@ -312,12 +312,18 @@ await client.notifications.markAllRead(); // through sys_inbox_message and tracks read-state in sys_notification_receipt. // These helpers will be repointed during the objectui bell cut-over. -// AI — AI-powered features +// AI — DECLARED BUT NOT IMPLEMENTED (#3718). These three methods build +// /api/v1/ai/{nlq,suggest,insights}, and no server in any repo mounts those +// paths, so every call 404s. They are typed and shipped, which is why they +// look usable; they are not. Do not build on them until #3718 resolves. await client.ai.nlq({ query: 'Show me all active accounts' }); await client.ai.suggest({ object: 'account', field: 'industry' }); await client.ai.insights({ object: 'sales', recordId: dealId }); -// Note: conversational chat is no longer on the client — use the Vercel AI SDK -// (`useChat()` from `@ai-sdk/react`) directly against your chat endpoint. +// Conversational chat is not on the client — use the Vercel AI SDK +// (`useChat()` from `@ai-sdk/react`) directly against the chat endpoint. +// That endpoint DOES exist (POST /api/v1/ai/chat, plus /chat/stream, +// /complete, /models and six /conversations routes, all served by service-ai); +// the SDK simply has no method for any of them. // i18n — Internationalization await client.i18n.getLocales(); diff --git a/content/docs/api/plugin-endpoints.mdx b/content/docs/api/plugin-endpoints.mdx index 1b74877521..e973f84595 100644 --- a/content/docs/api/plugin-endpoints.mdx +++ b/content/docs/api/plugin-endpoints.mdx @@ -95,14 +95,37 @@ The core dispatcher implements only the list / read / read-all routes above. Dev ### AI (`/ai`) — Plugin Required -| Method | Endpoint | Description | -|:-------|:---------|:------------| -| POST | `/ai/nlq` | Natural language → query | -| POST | `/ai/suggest` | Get value suggestions | -| POST | `/ai/insights` | Get data insights | +| Method | Endpoint | Description | Status | +|:-------|:---------|:------------|:-------| +| POST | `/ai/nlq` | Natural language → query | **Not implemented** | +| POST | `/ai/suggest` | Get value suggestions | **Not implemented** | +| POST | `/ai/insights` | Get data insights | **Not implemented** | - -There is no `/ai/chat` route — the AI chat route was removed so the wire protocol aligns with the Vercel AI SDK. Use `useChat()` (`@ai-sdk/react`) directly against the streaming chat endpoint rather than the client SDK's `ai` namespace, which intentionally does not expose a `chat` method. + +**This table was inverted, and is corrected here (#3718).** The three routes +above are declared in `DEFAULT_AI_ROUTES` and called by +`client.ai.nlq/suggest/insights`, but **no plugin in any repo mounts them** — +every call 404s. Conversely, this callout used to state "there is no +`/ai/chat` route", which was wrong: `/ai/chat` is mounted. + +What the AI plugin (`service-ai`) actually serves is a different set of +**12** routes: + +| Method | Endpoint | +|:-------|:---------| +| POST | `/ai/chat`, `/ai/chat/stream`, `/ai/complete` | +| GET | `/ai/models`, `/ai/status`, `/ai/effective-model` | +| POST / GET | `/ai/conversations` | +| GET / PATCH / DELETE | `/ai/conversations/:id` | +| POST | `/ai/conversations/:id/messages` | + +None of those has a client-SDK method — the SDK's `ai` namespace and the real +AI surface do not overlap at all. Reviewed dispositions for all 12 live in the +`cloud` repo, `packages/service-ai/src/ai-route-ledger.ts`. + +The original point stands on its own terms: use `useChat()` (`@ai-sdk/react`) +directly against the streaming chat endpoint, because the client SDK exposes no +`chat` method. ### i18n (`/i18n`) — Plugin Required diff --git a/content/docs/kernel/services-checklist.mdx b/content/docs/kernel/services-checklist.mdx index d4276eee66..84d27254ac 100644 --- a/content/docs/kernel/services-checklist.mdx +++ b/content/docs/kernel/services-checklist.mdx @@ -251,6 +251,14 @@ Trigger engine, event triggers from ObjectQL hooks, flow executor, scheduled tri ### 10. ai — 3 methods `aiNlq`, `aiSuggest`, `aiInsights` + +Declared only. All three are **optional** protocol methods (`aiNlq?` …) and no +service in any repo implements them, so the `/ai/{nlq,suggest,insights}` routes +they back are mounted by nothing and the matching `client.ai.*` calls 404 +(#3718). The AI service that does exist (`service-ai`) serves a different set +— chat, complete, models, conversations — through none of these methods. + + ### 11. i18n — 3 methods `getLocales`, `getTranslations`, `getFieldLabels` diff --git a/docs/audits/2026-07-dispatcher-client-route-coverage.md b/docs/audits/2026-07-dispatcher-client-route-coverage.md index 0a0c95e627..f6bfa7e36f 100644 --- a/docs/audits/2026-07-dispatcher-client-route-coverage.md +++ b/docs/audits/2026-07-dispatcher-client-route-coverage.md @@ -355,6 +355,50 @@ against the framework revision cloud builds and ships against. That is the right revision to check — but it means a `projects.*` change landing here is not verified against the control plane until that pin moves. +## 14. The last wildcard, and what it was actually worth (#3718) + +§10's capstone ratcheted "matched only by a `**` family" as *weaker* evidence, +to be driven down by enumerating each dynamic family. 60 → 3 after #3656. The +last 3 were `ai.nlq` / `ai.suggest` / `ai.insights` on `* /ai/**`. + +Enumerating that family answered the question the ratchet was really asking. +`service-ai` lives in the `cloud` repo (Cloud/EE), so this repo could never see +its table — the dispatcher just proxies to whatever `buildAIRoutes()` returned. +That table is **12 routes**: `chat`, `chat/stream`, `complete`, `models`, +`status`, `effective-model`, and six `conversations` routes. + +**None of them is `/nlq`, `/suggest` or `/insights`.** + +| | | +|---|---| +| `client.ai` methods | 3 — `nlq`, `suggest`, `insights` | +| …that any repo mounts | **0** | +| Real AI routes | 12 | +| …expressed by the SDK | **0** | + +The two sets are disjoint. So the wildcard was not weak evidence — it was +**wrong** evidence, certifying three URLs nothing serves, and its note even +asserted the client "expresses nlq/suggest/insights against the REST AI routes" +(never verified, false). `DEFAULT_AI_ROUTES` in `plugin-rest-api.zod.ts` +declares all three, but has **no runtime consumer** — only the spec's own test +reads it — and `aiNlq?` / `aiSuggest?` / `aiInsights?` are optional protocol +methods nothing implements. Declared, never built (#3718). Instances 7–9 of the +`the method exists ≠ the method can be called` class. + +`/api/v1/ai/` is now a bounded prefix exemption alongside the control plane — +two cross-repo surfaces, both ledgered in `cloud` +(`packages/service-ai/src/ai-route-ledger.ts`, 10 `gap` / 2 `server-only`, +enumerated straight off the array `buildAIRoutes()` returns). **The +wildcard-only bound is 0**, and the assertion is `toBe(0)` rather than a +ratchet: every matched call now rests on an exact enumerated route, and +reintroducing a `**` match reintroduces the one kind of evidence this audit +family has caught being wrong. + +The generalisable lesson, and the one worth carrying out of §1–§14: **a claim +about a family is not a claim about a member.** Wildcards, prefixes and "the +service handles that" all read as coverage in a green suite. Every one of them +this audit opened turned out to be hiding something. + ## Follow-up slicing (proposed) 1. **`client.actions.invoke(...)`** — closes the largest hole (3 routes). @@ -370,7 +414,8 @@ not verified against the control plane until that pin moves. 11. **Control-plane surface** (§10) — done in #3655; the ledger lives in `cloud` (§13), which is the only repo where both halves are in scope. 12. **Enumerate `/auth/**`** (§11) — done in #3656; wildcard ratchet 60 → 3. -13. **Enumerate `/ai/**`** — the last dynamic family, 3 SDK methods. +13. **Enumerate `/ai/**`** — done (§14). The wildcard ratchet is now **0**; the + 3 SDK methods turned out to be dead (#3718). 14. **Response-shape conformance** (§12) — error path done in #3675 for both services; the storage **success** bodies (three shapes, none carrying `success: true`) and the dispatcher's numeric `error.code` remain. diff --git a/packages/client/src/client-url-conformance.test.ts b/packages/client/src/client-url-conformance.test.ts index c9519def68..1fec2244e2 100644 --- a/packages/client/src/client-url-conformance.test.ts +++ b/packages/client/src/client-url-conformance.test.ts @@ -148,6 +148,29 @@ function matches(verb: string, path: string): Pattern | undefined { const CONTROL_PLANE = '/api/v1/cloud/'; const CONTROL_PLANE_NAMESPACE = 'projects.'; +/** + * The AI surface — the SECOND cross-repo prefix, and exempt for the same + * reason as the control plane rather than a different one. + * + * `/api/v1/ai/*` routes are built by `service-ai`'s `buildAIRoutes()` at plugin + * start, and `service-ai` is a Cloud/EE package living in the `cloud` repo. + * This repo's dispatcher only proxies to whatever that table contains (or 404s + * "AI service is not configured" when it is absent), so no ledger here can + * enumerate them — the same boundary `projects.*` sits behind. + * + * It used to be handled as a `* /ai/**` WILDCARD match instead, which was + * strictly worse: a wildcard says the family is claimed, so all three `ai.*` + * methods counted as matched. #3718 enumerated the real table in `cloud` and + * found the SDK's three URLs are not in it — `/nlq`, `/suggest` and + * `/insights` are mounted by nothing, in any repo (#3718). The wildcard was + * not weak evidence, it was wrong evidence, which is exactly why the ratchet + * below treats `**` matches as something to drive to zero rather than tolerate. + * + * Bounded the same way as the control plane: only `ai.*` may use the prefix. + */ +const AI_PLANE = '/api/v1/ai/'; +const AI_NAMESPACE = 'ai.'; + // --------------------------------------------------------------------------- // 2. The recorder // --------------------------------------------------------------------------- @@ -326,6 +349,7 @@ describe('client URL conformance ↔ the union of all four route ledgers (#3642) const silent: string[] = []; const malformed: string[] = []; const controlPlane: string[] = []; + const aiPlane: string[] = []; const wildcardOnly: string[] = []; for (const name of METHODS) { @@ -347,6 +371,7 @@ describe('client URL conformance ↔ the union of all four route ledgers (#3642) } const path = new URL(call.url, BASE).pathname; if (path.startsWith(CONTROL_PLANE)) { controlPlane.push(`${name} → ${call.verb} ${path}`); continue; } + if (path.startsWith(AI_PLANE)) { aiPlane.push(`${name} → ${call.verb} ${path}`); continue; } const hit = matches(call.verb, path); if (!hit) { unmatched.push(`${name} → ${call.verb} ${path}`); continue; } if (hit.route.includes('**')) wildcardOnly.push(`${name} → ${call.verb} ${path} (via ${hit.route})`); @@ -381,18 +406,35 @@ describe('client URL conformance ↔ the union of all four route ledgers (#3642) ).toEqual([]); expect(controlPlane.length, 'the projects namespace should still be reaching the control plane').toBeGreaterThan(0); + // Same bounding for the AI prefix. `ai.*` is the ONLY namespace allowed to + // use it; anything else reaching /api/v1/ai/ is a method that has wandered + // into a surface no in-repo ledger can vouch for. + const aiTrespassers = aiPlane.filter((e) => !e.startsWith(AI_NAMESPACE)); + expect( + aiTrespassers, + `non-ai methods targeting the AI plane, which service-ai owns in the cloud repo:\n${aiTrespassers.join('\n')}`, + ).toEqual([]); + expect(aiPlane.length, 'the ai namespace should still be reaching the AI plane').toBeGreaterThan(0); + // HOW STRONG IS THIS GUARD, HONESTLY. A `**` row asserts only that a prefix // family is CLAIMED, not that the specific URL resolves. That was this // guard's biggest weakness at #3642: 60 of ~196 matched calls rested on // nothing better, 54 of them on `* /auth/**`. #3656 enumerated better-auth's // real route table, so those now match exact rows and the bound fell 60 → 3. - // What remains is `* /ai/**`, whose routes service-ai builds at plugin start. - // Ratcheted: enumerating that family lowers it; nothing raises it without a - // deliberate decision. + // The last 3 were `ai.nlq/suggest/insights` on `* /ai/**`, and enumerating + // THAT family (#3718, in `cloud`, where service-ai lives) showed the + // wildcard had not been weak evidence but WRONG evidence: none of the three + // URLs is in the real table, and nothing in any repo mounts them (#3718). + // They are now handled by the AI_PLANE exemption above and pinned as dead + // on the cloud side, so this bound is 0. + // + // ZERO IS THE POINT: every remaining matched call rests on an exact route + // some ledger enumerated. Raising this bound reintroduces the one kind of + // evidence this audit family has caught being wrong. expect( wildcardOnly.length, 'methods matched only by a wildcard `**` family — weaker evidence than an exact ' + - `route. Enumerate a dynamic family to lower this bound; do not raise it:\n${wildcardOnly.join('\n')}`, - ).toBeLessThanOrEqual(3); + `route, and demonstrably able to be wrong (#3718). Enumerate the family instead:\n${wildcardOnly.join('\n')}`, + ).toBe(0); }); }); diff --git a/packages/runtime/src/route-ledger.ts b/packages/runtime/src/route-ledger.ts index 133271f568..62b77b4f05 100644 --- a/packages/runtime/src/route-ledger.ts +++ b/packages/runtime/src/route-ledger.ts @@ -36,7 +36,17 @@ * not a resolvable route. `* /auth/**` was the worst of it — 54 SDK methods * resting on a prefix claim — until #3656 enumerated better-auth's real table * into `plugin-auth/src/auth-route-ledger.ts`, dropping the guard's - * wildcard-only count 60 → 3. The `* /ai/**` row below is what remains. + * wildcard-only count 60 → 3. + * + * The last 3 were the `ai.*` methods on `* /ai/**`, and enumerating THAT + * family settled the question of how much a `**` row is worth: not "weaker + * evidence" but, in this case, WRONG evidence. `service-ai` (a Cloud/EE + * package in the `cloud` repo) mounts 12 routes and not one of them is the + * `/nlq` `/suggest` `/insights` the SDK calls — the wildcard had been + * certifying three URLs that nothing anywhere serves (#3718). The capstone now + * exempts `/api/v1/ai/` by prefix like the control plane, the real table is + * ledgered in `cloud`, and the wildcard-only bound is **0**: every matched call + * rests on an exact enumerated route. * * This module is runtime-internal (not exported from the package index): it is * the guard's data, not public API. Promotion to `@objectstack/spec` is a @@ -180,9 +190,9 @@ export const ROUTE_LEDGER: readonly RouteLedgerEntry[] = [ { route: '* /auth/**', domain: '/auth', disposition: 'sdk', client: 'auth.me', note: 'wholesale delegate to the auth service. Not enumerable route-by-route HERE, but no longer unenumerated: since #3656 plugin-auth/src/auth-route-ledger.ts carries the 55 SDK-reached routes plus the full mounted inventory, read off better-auth\'s live auth.api table' }, - // ── ai (dynamic route table) ────────────────────────────────────────────── + // ── ai (dynamic route table, owned by another repo) ─────────────────────── { route: '* /ai/**', domain: '/ai', disposition: 'dynamic', - note: 'routes come from service-ai buildAIRoutes() at plugin start; client expresses nlq/suggest/insights against the REST AI routes' }, + note: 'routes come from service-ai buildAIRoutes() at plugin start — service-ai is a Cloud/EE package in the `cloud` repo, so this repo cannot enumerate them and the dispatcher only proxies (or 404s "AI service is not configured"). Enumerated on the other side of that boundary since #3718: cloud packages/service-ai/src/ai-route-ledger.ts. The previous note here claimed the client "expresses nlq/suggest/insights against the REST AI routes"; that was never verified and is FALSE — nothing mounts those three paths (#3718)' }, // ── meta (legacy chain) ─────────────────────────────────────────────────── { route: 'GET /meta', domain: '/meta', disposition: 'sdk', client: 'meta.getTypes' },