diff --git a/.changeset/actions-as-ai-tools.md b/.changeset/actions-as-ai-tools.md new file mode 100644 index 0000000000..de27d0c6cc --- /dev/null +++ b/.changeset/actions-as-ai-tools.md @@ -0,0 +1,22 @@ +--- +'@objectstack/spec': patch +'@objectstack/service-ai': patch +--- + +feat(ai): actions opt in to being AI tools via an `ai:` block (ADR-0011) + +Realigns ADR-0011 with its original opt-in design. An Action becomes an +AI-callable tool only when its metadata sets `ai.exposed: true`, which requires +an explicit, LLM-facing `ai.description` (≥40 chars, distinct from the UI +`label`). There is no heuristic auto-exposure and no description derived from +the label — a clean break from the first implementation's opt-out `aiExposed` +flag, which is removed (no compatibility shim; the platform has not shipped). + +The `ai:` block also carries `category`, `paramHints` (per-parameter JSON-Schema +refinement), `outputSchema` (summarised into the tool description for chaining), +and `requiresConfirmation` (overrides the destructive-action HITL default). +`AIToolDefinition` is extended to carry `category` / `outputSchema` / `objectName` +/ `requiresConfirmation`. The `@objectstack/service-ai` bridge +(`action-tools.ts`) now gates on opt-in, merges `paramHints`, and emits a lint +warning when an exposed destructive-looking action asserts itself safe via +`ai.requiresConfirmation: false`. diff --git a/docs/adr/0011-actions-as-ai-tools.md b/docs/adr/0011-actions-as-ai-tools.md index 0925eb5671..d29631108a 100644 --- a/docs/adr/0011-actions-as-ai-tools.md +++ b/docs/adr/0011-actions-as-ai-tools.md @@ -1,16 +1,27 @@ # ADR-0011: Actions as AI Tools (Operational Parity) -**Status**: Draft (2026-05-24) -**Authors**: HotCRM (objectstack-ai/hotcrm) — surfacing requirements while planning v1.1 -**Consumers**: `@objectstack/service-ai`, `@objectstack/spec` (ui, ai), `@objectstack/runtime` (ActionRegistry), every app that ships actions +**Status**: Accepted — implemented (2026-06-04) +**Authors**: HotCRM (objectstack-ai/hotcrm) — surfaced the requirement; design finalised in framework +**Consumers**: `@objectstack/service-ai` (the bridge), `@objectstack/spec` (ui.action, contracts.ai-service), every app that ships actions --- ## TL;DR -Today, AI agents call **Tools** registered with `ToolRegistry`. Apps separately register business operations as **Actions** (`engine.registerAction(...)` / `*.action.ts`) for UI buttons and HTTP endpoints. The two registries don't talk to each other, so giving the Copilot a new capability means writing the same logic twice — once as an Action (for humans), once as a Tool (for the LLM), and keeping them in sync forever. +AI agents call **Tools**. Apps register business operations as **Actions** +(`*.action.ts`) for UI buttons and HTTP endpoints. Without a bridge, giving the +Copilot a capability means writing the logic twice — once as an Action (for +humans), once as a Tool (for the LLM) — and keeping them in sync forever. -This ADR proposes that **every Action opts in to being an AI Tool by adding a single `ai:` block to its metadata**. The runtime exposes opted-in Actions to agents through the existing `ToolRegistry` surface — no new bridge contract, no Tool duplicates. Result: **operational parity** — anything an admin can do, the Copilot can do, by virtue of calling the same Action with the same permissions, validation, audit, and transaction guarantees. +Every Action opts in to being an AI Tool by adding a single **`ai:` block** to +its metadata. The runtime exposes opted-in Actions to agents through the +existing tool registry — no Tool duplicates. Result: **operational parity** — +anything an admin can do, the Copilot can do, by calling the *same* Action with +the same permissions, validation, audit, and transaction guarantees. + +Exposure is **opt-in, default off**. `ai.exposed: true` is required, and it +forces an explicit, LLM-facing `ai.description`. There is **no heuristic +auto-exposure** and **no description derived from the UI label**. --- @@ -18,41 +29,90 @@ This ADR proposes that **every Action opts in to being an AI Tool by adding a si ### What HotCRM tried -HotCRM v1 shipped 6 hand-authored "skills" under `src/skills/` (lead qualification, case triage, customer 360, email drafting, revenue forecasting, live data). Each skill is a `defineSkill({ tools: [...] })` bundle of inline tools or `ToolRegistry` references. - -While planning v1.1, the team asked: *"We're a metadata-driven platform — do we really need bespoke skill code for each business operation?"* The first round of exploration produced a "meta-skill + per-object `ai:` block" design (record_advisor / record_synthesizer / etc.), which still required admins to maintain a side-channel AI DSL. - -The breakthrough observation: **Actions are already the metadata vocabulary for "what the system can do"**. They have a name, a label, a description, a typed parameter schema, permissions, audit logging, transactions, and an implementation (sandboxed body, API target, flow trigger, or registered handler). They're the same shape a Tool needs, minus a description aimed at an LLM. - -### Why this isn't just "wrap actions in tools" +HotCRM v1 shipped 6 hand-authored "skills" under `src/skills/`. Each skill is a +`defineSkill({ tools: [...] })` bundle of inline tools. While planning v1.1 the +team asked: *"We're a metadata-driven platform — do we really need bespoke skill +code for each business operation?"* + +The breakthrough: **Actions are already the metadata vocabulary for "what the +system can do".** They have a name, a label, a typed parameter schema, +permissions, audit logging, transactions, and an implementation. They're the +same shape a Tool needs, minus a description aimed at an LLM. + +### Design evolution (why this ADR was revised) + +The first implementation (commit `93c05899d`, the same day as the original +draft) took a shortcut that **inverted** this ADR's intent: a flat `aiExposed` +boolean with **heuristic auto-exposure** (expose anything that "looks safe") and +a tool **description derived from the UI label**. That contradicted two explicit +points of this ADR — opt-in-default-off is a *Goal*, and label-derived +descriptions are a *Non-Goal*. + +This revision realigns the code with the original opt-in `ai:` block design and +sharpens the rationale for the **AI-authoring era** (below). Because the platform +had **not yet shipped**, there were no live apps depending on auto-exposure — so +the realignment is a **clean break**: the `aiExposed` field is removed outright, +with no deprecation alias and no compatibility flag. + +### Why opt-in matters *more* when the authors are AI + +The platform's direction is that **actions are increasingly authored by AI** +(NL → Action; draft-then-review per ADR-0033). That reframes the trade-offs: + +* **"Writing a description is a burden" — gone.** An AI author writes a good + LLM-facing description for free. The friction that once argued for + auto-derivation evaporates; auto-derivation becomes *pointless*, not merely + low-quality. So we require an explicit `ai.description` and delete the + label-derivation default. +* **Opt-in is the governance gate, not a friction tax.** When AI both authors + *and* invokes actions, the platform's value to an enterprise is that a human + can govern *which* capabilities the agent fleet may invoke. `ai.exposed` + default-off is the physical boundary between "an AI drafted an action" and + "the agent fleet may now run it". A half-finished or unreviewed draft must + never be silently armed. +* **Schema minimalism is itself governance.** Every extra knob is one more way an + AI author misconfigures and one more thing a human reviewer must audit. So the + `ai:` block carries only fields with a real, end-to-end effect. + +### What separates a Tool from an Action-Tool -A naive bridge (auto-generate one Tool per Action) is *almost* right but has three sharp edges that this ADR resolves: - -1. **Not every Action should be AI-callable.** Internal actions (`__internal_delete_orphans`), destructive admin actions, and actions intended only for `list_toolbar` UX must not leak to the LLM. Opt-in is required. -2. **Action `params` are UI-oriented, not LLM-oriented.** `ActionParamSchema` carries `widget`, `placeholder`, i18n labels, etc. — useful for forms, noise for tool-calling. The bridge needs a clean translation to JSON Schema. -3. **The LLM-facing description is different from the human-facing label.** "Send Quote" (label) vs. "Send the latest approved quote PDF to the primary contact via email. Only call after the user has confirmed they want to send." (description). Mixing them confuses both audiences. +| Source | Lives where | Used for | Example | +| --- | --- | --- | --- | +| **ToolRegistry** (pure tools) | platform / app code | Generic, schema-discoverable operations that aren't business actions | `describe_object`, `query_data` | +| **Action `ai:` block (this ADR)** | app metadata (`*.action.ts`) | App-specific business operations humans also invoke from the UI | `crm_case_triage`, `complete_task` | +| **SkillRegistry** | platform / app code | Bundles of *instructions* + a curated tool subset | `lead_qualification` | -A small, opt-in `ai:` extension on `ActionSchema` solves all three. +The `label` vs `ai.description` split survives the AI-authoring shift because the +*consumption* audience is still mixed: humans click UI buttons (`label`), agents +call tools (`ai.description`). Only the authoring side became AI. --- ## Goals -* **Zero duplication** — one definition (the Action) drives both the UI button and the AI tool. -* **Opt-in exposure** — actions are NOT exposed to the LLM by default; the `ai.exposed: true` flag is required. -* **Strong description for LLMs** — separate `ai.description` from the human `label`, so model prompts can be tuned without changing the UI. -* **Auto-derived JSON Schema** — `parameters` for the AI tool come from the Action's `params[]` and the referenced field types, with admin-supplied overrides for AI-only refinement (e.g. tighter `enum`, longer `description`). -* **Permission parity** — the AI invocation goes through the same permission and audit machinery as the human invocation. -* **Confirmation parity** — actions that require `confirmText` in the UI surface `requiresConfirmation: true` to the agent runtime; the LLM proposes, the user accepts. -* **Backwards compatible** — existing `defineTool(...)` and `defineSkill(...)` continue to work; this ADR adds a third source of tools alongside them. +* **Zero duplication** — one definition (the Action) drives both the UI button + and the AI tool. +* **Opt-in exposure, default off** — `ai.exposed: true` is required; this is the + governance gate. +* **Explicit LLM description** — `ai.description` is required when exposed, + authored for a model, never derived from `label`. +* **Auto-derived JSON Schema** — tool `parameters` come from the Action's + `params[]` + referenced field types, refined by `ai.paramHints`. +* **Permission parity** — AI invocation goes through the same permission/RLS/audit + machinery as human invocation. +* **Confirmation parity** — destructive actions route through the HITL approval + queue; `ai.requiresConfirmation` lets the author override the default. ## Non-Goals -* Replacing `ToolRegistry`. Pure tools (`describe_object`, `query_records`, etc.) remain first-class — they're not business operations and don't belong in a UI's action surface. -* Auto-generating an LLM `description` from the action label. Authors who want AI exposure must write a description aimed at a model. -* Inferring "this action is safe / unsafe" automatically. The author opts in and writes the confirmation copy. -* Touching `SkillRegistry`. Skills (instruction bundles + tool filters) are a separate concept that this ADR clarifies (see "Relationship" below) but does not change. -* Hot-reload semantics. Live-reload of newly added actions into running agent sessions is a separate concern handled by ADR-0008/0009 metadata watchers; this ADR only requires that the next agent turn sees the current registry. +* Replacing `ToolRegistry`. Pure tools stay first-class. +* Auto-generating an LLM `description` from the label. (Authors — human or AI — + write it.) +* Inferring "safe / unsafe" automatically. The author opts in and sets the + confirmation policy. +* Touching `SkillRegistry`. +* Hot-reload of newly-added actions into running sessions (tool definitions are + resolved per turn — see Open Questions). --- @@ -60,162 +120,98 @@ A small, opt-in `ai:` extension on `ActionSchema` solves all three. ### 1. Action spec extension — `ai:` block -Add an optional `ai` field to `ActionSchema` in `@objectstack/spec/src/ui/action.zod.ts`: +`packages/spec/src/ui/action.zod.ts` gains an optional `ai` field +(`ActionAiSchema`): ```ts -const ActionAiSchema = z.object({ - /** - * Expose this action to AI agents as a callable tool. Default false. - * Setting this to true REQUIRES `description` (so the LLM knows when to call). - */ +export const ActionAiSchema = z.object({ + /** Expose to AI agents. Default false. Requires `description` when true. */ exposed: z.boolean().default(false), - /** - * LLM-facing description. Tells the model when and why to call this action. - * Distinct from `label` (which is UI-facing and i18n-translated). - * Should be plain English and ≥ 40 chars for useful tool selection. - */ + /** LLM-facing description (≥40 chars). Required when exposed. Distinct from `label`. */ description: z.string().min(40).optional(), - /** - * Override tool category. Defaults to 'action' (side-effect). - * Set to 'data' for read-only actions, 'analytics' for aggregations, etc. - */ - category: ToolCategorySchema.optional(), - - /** - * Per-parameter AI hints, keyed by param name. Tightens the JSON Schema - * the LLM sees (e.g. add `enum`, override `description`) without changing - * the UI-facing field metadata. - */ + /** Tool category override. Defaults to 'action' (side-effect). */ + category: ActionAiCategorySchema.optional(), // mirrors ToolCategorySchema + + /** Per-parameter AI hints (tighter enum / description / examples), keyed by param name. */ paramHints: z.record(z.string(), z.object({ description: z.string().optional(), enum: z.array(z.union([z.string(), z.number()])).optional(), examples: z.array(z.unknown()).optional(), })).optional(), - /** - * Output JSON Schema for the action's return value. Enables structured - * downstream tool chaining (one action's output feeds another's input). - * Optional — when omitted the LLM treats the return value as freeform. - */ + /** Output JSON Schema — enables downstream chaining; summarised into the description. */ outputSchema: z.record(z.string(), z.unknown()).optional(), - /** - * Override `requiresConfirmation` for AI calls. Defaults to true when the - * action has `confirmText` set OR `type` is 'delete'. Set explicitly to - * false for clearly safe actions like 'send_test_email'. - */ + /** Override HITL confirmation. Defaults to true for destructive-looking actions. */ requiresConfirmation: z.boolean().optional(), -}).optional(); +}); ``` -**Validation rules** (added to `ActionSchema.refine(...)`): +The category enum is **inlined** (not imported from `ai/tool.zod`) to avoid a +`ui → ai` import cycle (`ai/*.form.ts` already imports from `ui/view.zod`). -* If `ai.exposed === true`, `ai.description` is required. -* If `ai.exposed === true` and any `params[].field` is set, all referenced fields must be readable by the action's caller (enforced at registration, not parse). -* `ai.paramHints` keys must match a `params[].name` (validated at refine). +**Validation rules** (`ActionSchema.refine`): -### 2. Registry bridge — `ActionRegistry → ToolRegistry` +* `ai.exposed === true` ⇒ `ai.description` required. +* `ai.paramHints` keys must match a declared `params[].name` (or the injected + `recordId`) — a typo can't silently no-op. -The runtime's `ActionRegistry` (currently exposed via `engine.registerAction(...)` and the dispatcher) gains a single method: +The old flat `aiExposed` boolean is **removed**. -```ts -interface ActionRegistry { - // existing - register(action: Action, handler: ActionHandler): void; - list(filter?: ActionFilter): Action[]; - - // NEW — derives AIToolDefinitions from registered actions that opt in. - toolsForAi(opts: { - user: User; - objectName?: string; // optional scope filter - }): AIToolDefinition[]; -} -``` +### 2. The bridge — `@objectstack/service-ai/src/tools/action-tools.ts` -`toolsForAi` walks `list()`, keeps actions where `ai.exposed === true`, filters by the user's permissions (re-using whatever check the action invoker uses), and translates each one to an `AIToolDefinition`: +The bridge (not a `runtime.ActionRegistry` method, as the first draft sketched) +walks every object's `actions[]` via the metadata service and registers the +opted-in ones into the existing `ToolRegistry`: -| Source on Action | Target on Tool | Notes | -| --- | --- | --- | -| `name` | `name` | `${objectName}__${actionName}` when objectName set, else `name` | -| `ai.description` | `description` | required, validated at action-parse time | -| `ai.category ?? 'action'` | `category` | | -| `params[]` + `ai.paramHints` | `parameters` | JSON Schema generated from field types (already done for HTTP dispatch); paramHints merged last | -| `ai.outputSchema` | `outputSchema` | | -| `objectName` | `objectName` | | -| `ai.requiresConfirmation ?? !!confirmText \|\| type==='delete'` | `requiresConfirmation` | | -| `permissions` | `permissions` | passes through | -| `active` | `active` | inherited | -| (constant) `true` | `builtIn` | false — these are app-defined, not platform tools | +* **`actionSkipReason(action, ctx)`** — opt-in gate. Skips unless + `ai.exposed === true` (and `ai.description` present). Then the structural and + wiring checks (UI-only types, missing target/body, no apiClient/automation), + then the destructive-action gate. +* **`actionToToolDefinition(...)`** — builds the `AIToolDefinition`: + `description` from `ai.description` (+ a compact `Returns: …` line summarising + `ai.outputSchema`), `parameters` from `params[]` refined by `ai.paramHints`, + and carries `category` / `outputSchema` / `objectName` / `requiresConfirmation`. +* **`actionRequiresApproval(action)`** — `ai.requiresConfirmation` wins; else the + heuristic (`confirmText`, `mode:'delete'`, `variant:'danger'`). +* **Lint guardrail** — when an exposed action *looks* destructive yet the author + set `ai.requiresConfirmation:false`, it registers (the author's call) but the + bridge emits a `warning` the plugin logs. -The dispatcher already knows how to translate `params[]` into a JSON Schema for the HTTP layer (`ActionParamSchema.field` → resolve field type → emit property). The bridge reuses that machinery. +`AIToolDefinition` (`packages/spec/src/contracts/ai-service.ts`) was extended with +optional `category`, `outputSchema`, `objectName`, `requiresConfirmation` so this +richer info is carried rather than dropped. ### 3. Agent-runtime integration -`packages/services/service-ai/src/agent-runtime.ts` currently composes `availableTools` from `ToolRegistry` only. Change: - -```ts -// Before -const availableTools = toolRegistry.list(); - -// After -const availableTools = [ - ...toolRegistry.list(), // platform + app-registered tools - ...actionRegistry.toolsForAi({ user: ctx.user }), // exposed actions -]; -``` - -Agent tool resolution (`agent.tools[]` lookup, skill flattening) stays unchanged — it just sees more candidates in `availableTools`. - -**Name collision rule**: if a tool with the same `name` is registered in both `ToolRegistry` and via `ActionRegistry`, the bridge logs a warning and the `ToolRegistry` entry wins. This is so platform tools can never be silently overridden by an app's action. +No change needed beyond registration: action-tools register into the same +`ToolRegistry` the plugin already feeds into `availableTools`. The agent's tool +resolution sees them as candidates automatically. Pure tools win name +collisions (an app action can never silently override a platform tool — the +registry refuses a duplicate name). ### 4. Invocation flow -When an LLM decides to call an action-backed tool: - ``` -LLM emits tool_call { name: 'crm_case__triage', args: { caseId: '00123' } } - │ - ▼ -agent-runtime resolves tool definition → finds it has `objectName: 'crm_case'`, - is action-backed (set by the bridge as a marker on AIToolDefinition.meta.kind='action') - │ - ▼ - if requiresConfirmation === true: - runtime returns a `pending_action` event to the client - UI renders the action confirmation card (re-uses existing action confirm UI) - user clicks Accept → client POSTs to /api/v1/.../actions/triage with the same args - else: - runtime invokes ActionRegistry.execute(action, args, { user, source: 'ai' }) +LLM emits tool_call { name: 'action_complete_task', args: { recordId: '00123' } } │ - ▼ -ActionRegistry executes the action body (sandboxed JS / API / flow) inside the -same transaction + audit machinery used for human-invoked actions. -The `source: 'ai'` flag is recorded on the audit log entry. + ▼ if requiresConfirmation && HITL wired: + handler enqueues a pending action → returns { status: 'pending_approval' } + operator approves in Studio → the pre-registered bypass dispatcher runs it + else: + handler dispatches by type — script → dataEngine.executeAction; + api → apiClient.request; flow → automation.execute │ - ▼ -Result returned to the LLM as the tool's return value (shape validated against -`ai.outputSchema` if provided). + ▼ executes inside the same RLS / permission / audit machinery as a human click. + Result returned to the LLM (shape documented by ai.outputSchema if set). ``` -**Key invariant**: AI-invoked actions never bypass permission checks, validation rules, hooks, or audit. The only differences vs. human invocation are (a) the input args came from an LLM, not a form, and (b) `source: 'ai'` is stamped on the audit entry. - ---- - -## Relationship to ToolRegistry and SkillRegistry - -This ADR adds a **third source** of tool definitions; the three sources remain distinct: - -| Source | Lives where | Used for | Example | -| --- | --- | --- | --- | -| **`ToolRegistry`** | platform / app code | Pure, generic, schema-discoverable operations that aren't business actions | `describe_object`, `query_records`, `aggregate_data`, `propose_flow` (ADR-0010) | -| **`ActionRegistry` (via this ADR)** | app metadata (`*.action.ts`) | App-specific business operations that humans also invoke from the UI | `crm.lead.qualify`, `crm.case.triage`, `crm.account.send_quote` | -| **`SkillRegistry`** | platform / app code | Bundles of *instructions* + a curated tool subset; activates conditionally | `lead_qualification` skill = "be a BANT expert" + tools `[query_records, crm.lead.qualify]` | - -Skills remain useful even after this ADR — they're **how you bias an agent toward calling certain action-tools in certain contexts**. `defineSkill({ tools: ['crm.case.triage'] })` becomes the idiomatic way to surface action-tools. - -**HotCRM's planned migration** (informative, not normative for this ADR): delete the 6 hand-authored skills in `src/skills/`, replace each with a `defineAction({ ai: { exposed: true } })`, optionally keep a minimal skill bundle that pre-loads instructions for sales vs. service personas while letting the agent see *all* exposed actions. +**Invariant**: AI-invoked actions never bypass permissions, validation, hooks, or +audit. The only differences vs a human click are the args came from an LLM and +the invocation is attributed to the chat actor (falling back to a synthetic +`ai_agent` principal). --- @@ -223,174 +219,98 @@ Skills remain useful even after this ADR — they're **how you bias an agent tow ### Permissions -Today, action execution permission is checked at two layers: - -1. **HTTP dispatcher** — verifies the caller's session has `action.permissions` (or default object-write) before invoking the handler. -2. **ObjectQL** — the handler runs as the caller; sharing rules and FLS apply to any reads/writes inside. - -AI invocations reuse **both** layers. The `User` passed to `actionRegistry.toolsForAi({ user })` is the chat session's user; the LLM cannot escalate by asking. The bridge filters `toolsForAi` output to actions the user is permitted to invoke — actions the user can't call are simply not visible to the LLM (avoids the LLM trying and getting a permission-denied error mid-conversation). +The bridge runs each action under the chat session's user (or a synthetic +`ai_agent` principal). RLS / FLS / sharing rules apply to every read and write +inside the handler exactly as for a human click. The LLM cannot escalate by +asking; an action the user can't invoke simply isn't offered. -### Confirmation +### Confirmation (HITL approval queue) -Three states: - -| Action config | LLM tool sees | Behavior | -| --- | --- | --- | -| `confirmText` set OR `type: 'delete'` | `requiresConfirmation: true` | LLM proposes; runtime emits `pending_action`; user clicks Accept in UI | -| `ai.requiresConfirmation: true` (override) | `requiresConfirmation: true` | same | -| Neither, no destructive type | `requiresConfirmation: false` | LLM calls directly; result returned to LLM for next step | -| `ai.requiresConfirmation: false` on a destructive action | (override applies) | proceeds without confirmation — **author asserts this is safe** | - -For HotCRM specifically: anything that writes to `status`, `stage`, `amount`, or fires an email should keep `requiresConfirmation: true`. Customer 360 / forecast / triage *suggestions* (which don't write until accepted) can run without confirmation because their effect is "show the user this draft." +| Action config | Behavior | +| --- | --- | +| destructive (`confirmText` / `mode:'delete'` / `variant:'danger'`) **and** HITL wired (`enableActionApproval` + `aiService`) | registered; invocation enqueues a pending action and returns `pending_approval` | +| destructive **and** HITL **not** wired | skipped (can't run unattended) | +| `ai.requiresConfirmation: true` on a safe action | treated as destructive (gated) | +| `ai.requiresConfirmation: false` on a destructive action | exposed & direct-run — **author asserts safe** (bridge logs a warning) | +| not destructive | direct-run | --- ## Migration Path -### Phase 1 — framework (this ADR) - -1. Add `ai` block to `ActionSchema` (spec change, no runtime impact yet). -2. Implement `ActionRegistry.toolsForAi(...)` and the param→JSON-Schema translator. -3. Wire `agent-runtime` to merge `actionRegistry.toolsForAi(...)` into `availableTools`. -4. Add `meta.kind = 'action'` marker on AIToolDefinitions originating from the bridge, plus a runtime-side dispatcher: when the LLM calls an `kind === 'action'` tool, execute through `ActionRegistry`, not the inline handler. -5. Audit log enhancement: stamp `source: 'ai' | 'human' | 'system'` on every action invocation entry. -6. Tests: action exposes correctly; permission filter respected; confirmation flow surfaces `pending_action`; output schema validation. - -### Phase 2 — first consumer (HotCRM v1.1) +### Phase 1 — framework (this ADR) — **done** -1. Convert `src/skills/case-triage.skill.ts` → `src/actions/crm_case_triage.action.ts` with `ai: { exposed: true, description: '...' }`. Verify the Copilot calls it. -2. Convert the remaining 5 business skills the same way. -3. Delete `src/skills/` (or trim it to a sales/service persona instruction bundle). -4. Update `content/docs/ai-copilot/skills.mdx` → rename to `actions.mdx`, restructure around the action catalog. +1. `ai` block on `ActionSchema`; `aiExposed` removed (clean break). +2. `AIToolDefinition` extended (`category` / `outputSchema` / `objectName` / + `requiresConfirmation`). +3. Bridge rewritten to opt-in: `actionSkipReason`, `actionToToolDefinition`, + `actionRequiresApproval`, paramHints merge, outputSchema summary, lint + warnings. +4. Studio `action.form` updated (`aiExposed` field → `ai` block). +5. Internal `ai_pending_actions` approve/reject actions: dropped the now-redundant + `aiExposed:false` (opt-in means human-only by default). +6. Tests: spec `ai` block validation; bridge opt-in gating, paramHints, + outputSchema, category, requiresConfirmation override, lint warning. -### Phase 3 — ecosystem - -1. Document the pattern in `docs/guides/ai-actions.md` with a 5-minute "write an AI-callable action" tutorial. -2. Add a Stack Lint rule: `actions with type: 'delete' must explicitly set ai.exposed` (force authors to make a deliberate choice). -3. Optional: a `defineAiAction(...)` factory that defaults `ai.exposed: true` for authors who want a shorter idiom. - ---- +### Phase 2 — first consumer — **done (app-todo testbed)** -## Schema Changes +The `examples/app-todo` script actions (`complete_task`, `start_task`, +`clone_task`, `mass_complete`, `export_csv`) opt in with `ai.exposed` + +`ai.description`; `delete_completed` opts in but stays destructive (registers +only when HITL approval is wired); the modal actions stay UI-only. The +`test:action` / `test:hitl` demos exercise the path end-to-end. -### `@objectstack/spec` — additive only +HotCRM v1.1 follows: convert the 6 hand-authored skills to AI-exposed actions, +keeping a thin sales/service persona instruction bundle. -* `packages/spec/src/ui/action.zod.ts` — add `ai` field (see Design §1). -* `packages/spec/src/ai/tool.zod.ts` — no change required; `'action'` category already exists. -* `packages/spec/src/ai/agent.zod.ts` — no change. `agent.tools[]` references work as-is; agents that want to allow-list action-tools just include their names there. - -### Type exports - -```ts -// new exports -export type ActionAi = z.infer; -export type ActionWithAi = Action & { ai: ActionAi & { exposed: true } }; -``` - -### Examples (HotCRM, illustrative) - -```ts -// src/actions/crm_case_triage.action.ts -import { defineAction } from '@objectstack/spec'; - -export default defineAction({ - name: 'crm_case_triage', - label: { en: 'Triage Case', 'zh-CN': '案例分类' }, - objectName: 'crm_case', - type: 'script', - body: { language: 'js', source: '...', capabilities: ['ai', 'objectql'] }, - params: [ - { field: 'id', objectOverride: 'crm_case', required: true }, - ], - ai: { - exposed: true, - description: - 'Classify a support case: suggest priority (P0–P3), category (billing/technical/account), ' + - 'and queue. Reads the case subject, description, account tier, and prior cases. ' + - 'Returns a suggestion — does not write until the user confirms.', - outputSchema: { - type: 'object', - properties: { - priority: { type: 'string', enum: ['P0', 'P1', 'P2', 'P3'] }, - category: { type: 'string', enum: ['billing', 'technical', 'account'] }, - queue: { type: 'string' }, - confidence: { type: 'number' }, - }, - required: ['priority', 'category'], - }, - requiresConfirmation: false, // it returns a draft; user accepts via a downstream action - }, -}); -``` +### Phase 3 — ecosystem -```ts -// src/actions/crm_case_resolve.action.ts — destructive, defaults to confirmation -export default defineAction({ - name: 'crm_case_resolve', - label: { en: 'Resolve Case' }, - objectName: 'crm_case', - type: 'script', - confirmText: { en: 'Mark this case resolved and notify the customer?' }, - // confirmText set → ai.requiresConfirmation defaults to true; no override needed - params: [ - { field: 'id', objectOverride: 'crm_case', required: true }, - { name: 'resolution_summary', type: 'longtext', required: true }, - ], - ai: { - exposed: true, - description: - 'Mark a case resolved with a final resolution summary. Sends a notification email to the ' + - 'primary contact. Only call after the user has agreed the case is solved.', - }, -}); -``` +1. Guide: "write an AI-callable action" (5-minute tutorial). +2. Stack Lint: `type:'delete'` actions must make a deliberate `ai.exposed` + choice. +3. Optional `defineAiAction(...)` sugar (defaults `exposed:true`) — low priority + since AI authors write the block in full. --- ## Open Questions -1. **Bulk vs single.** An action with `bulkEnabled: true` accepts an array of record IDs from the UI. Should the AI tool expose two signatures (single + bulk) or one (always-array)? **Tentative**: one tool with an array-accepting `parameters`, since LLMs handle this well. - -2. **Streaming results.** Long-running actions (a forecast over thousands of opps) might want to stream partial results back to the LLM for early reasoning. Out of scope here; would need a separate ADR on streaming tool returns. - -3. **Tool name namespacing.** Proposed: `${objectName}__${actionName}` (double underscore) for object-bound actions, bare `actionName` for global ones. Alternative: a `.` separator. Decided in favor of `__` because tool names are constrained to `^[a-z_][a-z0-9_]*$` today; changing the regex is a larger spec change. - -4. **MCP exposure.** Should action-tools also be exposed via the existing MCP server plugin (`@objectstack/plugin-mcp-server`)? Likely yes, but uses the same `toolsForAi(...)` output — covered transparently by reusing the bridge. Confirm in implementation. - -5. **Discoverability for the LLM.** With 50+ actions in a mature app, the LLM's context fills up with tool definitions. Mitigations: (a) scope `toolsForAi` by current record's object on detail pages, (b) skills as curators for global chat, (c) lazy tool resolution (let the LLM ask "what actions exist for crm_case?"). Implementation can start without these and add as needed. - -6. **Versioning.** When an action's `params` or `outputSchema` changes, ongoing conversations may have already seen the old shape. Strategy: tool definitions are resolved per turn (not per conversation), so the next turn sees the new shape automatically. Mid-turn schema drift is a non-issue because tool resolution and invocation happen in the same turn. - ---- - -## Out of Scope - -* **Replacing Tools.** Schema-derived tools (`describe_object` et al.) are not actions and stay in `ToolRegistry`. -* **Action chaining DAGs.** "Run A, then B with A's output" is the LLM's job, not a separate composition layer. -* **Side-effect simulation / dry-run.** Useful for "show me what would happen" but out of scope here; can be added per-action with a `dryRun: true` parameter convention. -* **AI-authored actions.** "The Copilot wrote a new action" is ADR-0010's territory (NL → Flow), with the analogous extension to NL → Action covered separately. -* **Per-conversation tool subsetting based on user history.** Personalized tool catalogs are an optimization, not a correctness concern. +1. **Bulk vs single.** Tentative: one tool with an array-accepting `parameters`. +2. **Streaming results.** Out of scope; needs a streaming-tool-return ADR. +3. **Tool name namespacing.** Implemented as a prefix: `action_`. +4. **MCP exposure.** Action-tools live in the same `ToolRegistry`, so the MCP + server plugin surfaces them transparently. Confirm in MCP work. +5. **Discoverability with 50+ actions.** Mitigations: scope `toolsForAi` by the + current object, skills as curators, lazy tool resolution. Add as needed. +6. **Versioning.** Tool definitions resolve per turn, so the next turn sees the + current shape; mid-turn drift is a non-issue. +7. **Audit attribution (tri-state).** Today invocations are attributed to the + actor (or synthetic `ai_agent`). A richer `source: 'ai-authored' | 'ai-invoked' + | 'human'` distinction — separating "an AI *wrote* this action" from "an AI + *called* it" — pairs with the ADR-0033 / NL→Action authoring work. --- ## Decision -Adopt the design above for `@objectstack/spec` v6.x. Implementation lives in: +Adopt the opt-in `ai:` block. Implementation: -* `@objectstack/spec` — additive `ai` block on `ActionSchema` -* `@objectstack/runtime` — `ActionRegistry.toolsForAi(...)` + dispatcher routing for action-kind tools -* `@objectstack/service-ai` — merge action-tools into `availableTools` in `agent-runtime` +* `@objectstack/spec` — `ActionAiSchema` on `ActionSchema`; `AIToolDefinition` + extension; `action.form` update. +* `@objectstack/service-ai` — `action-tools.ts` bridge + HITL approval queue + (already shipped) reading the `ai:` block. -HotCRM v1.1 ships as the first consumer and the validation testbed. +`examples/app-todo` is the first consumer and validation testbed. HotCRM v1.1 +ships next. --- ## References * ADR-0003 — Package as first-class citizen (where actions live) -* ADR-0008 — Metadata repository and change log (how registry mutations propagate) -* ADR-0009 — Execution-pinned metadata (which version of an action a tool call uses) +* ADR-0008 / ADR-0009 — Metadata change log & execution-pinned metadata * ADR-0010 — Natural-language → Flow authoring (the AI-authoring counterpart) -* `packages/spec/src/ui/action.zod.ts` — existing `ActionSchema` -* `packages/spec/src/ai/tool.zod.ts` — existing `ToolSchema` -* `packages/services/service-ai/src/agent-runtime.ts` — current tool resolution +* ADR-0033 — Draft-gating for AI-written metadata (review-before-publish) +* `packages/spec/src/ui/action.zod.ts` — `ActionSchema` + `ActionAiSchema` +* `packages/spec/src/contracts/ai-service.ts` — `AIToolDefinition` +* `packages/services/service-ai/src/tools/action-tools.ts` — the bridge diff --git a/examples/app-todo/src/actions/task.actions.ts b/examples/app-todo/src/actions/task.actions.ts index 57abcf1d96..bf5405cd75 100644 --- a/examples/app-todo/src/actions/task.actions.ts +++ b/examples/app-todo/src/actions/task.actions.ts @@ -13,6 +13,10 @@ export const CompleteTaskAction: Action = { locations: ['record_header', 'list_item'], successMessage: 'Task marked as complete!', refreshAfter: true, + ai: { + exposed: true, + description: 'Mark a todo task as complete. Use when the user says a task is done or finished.', + }, }; /** Mark Task as In Progress */ @@ -26,6 +30,10 @@ export const StartTaskAction: Action = { locations: ['record_header', 'list_item'], successMessage: 'Task started!', refreshAfter: true, + ai: { + exposed: true, + description: 'Mark a todo task as in progress. Use when the user says they are starting or working on a task.', + }, }; /** Defer Task */ @@ -87,6 +95,10 @@ export const CloneTaskAction: Action = { locations: ['record_header'], successMessage: 'Task cloned successfully!', refreshAfter: true, + ai: { + exposed: true, + description: 'Duplicate an existing todo task, copying its fields into a new task record.', + }, }; /** Mass Complete Tasks */ @@ -100,6 +112,10 @@ export const MassCompleteTasksAction: Action = { locations: ['list_toolbar'], successMessage: 'Selected tasks marked as complete!', refreshAfter: true, + ai: { + exposed: true, + description: 'Mark all currently selected todo tasks as complete in one bulk operation.', + }, }; /** Delete Completed Tasks */ @@ -118,6 +134,13 @@ export const DeleteCompletedAction: Action = { confirmText: 'Permanently delete all completed tasks? This cannot be undone.', successMessage: 'Completed tasks deleted!', refreshAfter: true, + ai: { + exposed: true, + description: + 'Permanently delete every completed todo task. Destructive and irreversible — only after the user confirms.', + // confirmText + variant:'danger' default this to requiring HITL approval; + // it registers only when enableActionApproval is on, then routes to the queue. + }, }; /** Export Tasks to CSV */ @@ -131,4 +154,8 @@ export const ExportToCsvAction: Action = { locations: ['list_toolbar'], successMessage: 'Export completed!', refreshAfter: false, + ai: { + exposed: true, + description: 'Export the current list of todo tasks to a downloadable CSV file.', + }, }; diff --git a/examples/app-todo/test/ai-action.test.ts b/examples/app-todo/test/ai-action.test.ts index a745ce757d..48de028ffa 100644 --- a/examples/app-todo/test/ai-action.test.ts +++ b/examples/app-todo/test/ai-action.test.ts @@ -1,9 +1,10 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. // // AI **action** integration demo — the write-side counterpart to -// `ai-agent.test.ts`. Confirms that every `type: 'script'` action on -// the Task object is auto-registered as an `action_` tool, and -// that the `data_chat` agent can pick the right one in plain English. +// `ai-agent.test.ts`. Confirms that every `type: 'script'` action on the +// Task object that opts in via `ai.exposed` (ADR-0011) is registered as an +// `action_` tool, and that the `data_chat` agent can pick the right +// one in plain English. // // Run via: `pnpm --filter @example/app-todo test:action` // @@ -189,7 +190,7 @@ import { registerTaskActionHandlers } from '../src/actions/register-handlers'; } console.log('\n🎉 Action Demo Successful!'); - console.log(' • Script-type actions auto-exposed as `action_*` tools'); + console.log(' • Opted-in script actions (ai.exposed) registered as `action_*` tools'); console.log(' • Agent routed user request to action_complete_task'); console.log(' • Task status mutated from incomplete → completed'); console.log(' • chat_with_tools trace persisted in ai_traces'); diff --git a/packages/services/service-ai/src/__tests__/action-tools.test.ts b/packages/services/service-ai/src/__tests__/action-tools.test.ts index 70752d0228..1a84e2ecac 100644 --- a/packages/services/service-ai/src/__tests__/action-tools.test.ts +++ b/packages/services/service-ai/src/__tests__/action-tools.test.ts @@ -4,6 +4,7 @@ import { describe, it, expect, vi } from 'vitest'; import type { Action } from '@objectstack/spec/ui'; import { actionSkipReason, + actionToToolDefinition, buildApiRequestBody, createFetchApiClient, registerActionsAsTools, @@ -11,6 +12,8 @@ import { } from '../tools/action-tools.js'; import { ToolRegistry } from '../tools/tool-registry.js'; +// Actions are AI-exposed only by opt-in (ADR-0011), so the baseline fixture +// carries a valid `ai` block. Tests that exercise the opt-in gate override it. const baseAction = (over: Partial = {}): Action => ({ name: 'do_thing', @@ -19,6 +22,7 @@ const baseAction = (over: Partial = {}): Action => target: 'doThingHandler', objectName: 'task', locations: ['record_header'], + ai: { exposed: true, description: 'Do the thing the user asked for on this task record.' }, ...over, }) as Action; @@ -52,8 +56,89 @@ describe('actionSkipReason', () => { expect(actionSkipReason(baseAction({ variant: 'danger' }))).toMatch(/danger/); }); - it('respects aiExposed:false', () => { - expect(actionSkipReason(baseAction({ aiExposed: false }))).toMatch(/aiExposed/); + it('is opt-in: skips actions that did not set ai.exposed', () => { + expect(actionSkipReason(baseAction({ ai: undefined }))).toMatch(/not AI-exposed/); + expect(actionSkipReason(baseAction({ ai: { exposed: false } as never }))).toMatch(/not AI-exposed/); + }); + + it('skips an exposed action missing a description (defensive)', () => { + expect( + actionSkipReason(baseAction({ ai: { exposed: true } as never })), + ).toMatch(/description is missing/); + }); + + it('ai.requiresConfirmation:false lets an exposed destructive action run', () => { + // delete looks destructive, but the author asserts it is safe → exposed. + expect( + actionSkipReason(baseAction({ + mode: 'delete', + ai: { exposed: true, description: 'Archive this task record; it is reversible from trash.', requiresConfirmation: false }, + })), + ).toBeNull(); + }); + + it('ai.requiresConfirmation:true gates an otherwise-safe action behind HITL', () => { + const a = baseAction({ + ai: { exposed: true, description: 'Update the task title to the value the user supplied.', requiresConfirmation: true }, + }); + expect(actionSkipReason(a)).toMatch(/requires confirmation/); + expect( + actionSkipReason(a, { + enableActionApproval: true, + aiService: { proposePendingAction: async () => ({ id: 'x' }) }, + }), + ).toBeNull(); + }); +}); + +describe('actionToToolDefinition — ai: block translation', () => { + it('returns null when not exposed', () => { + expect(actionToToolDefinition(baseAction({ ai: undefined }), undefined, new Map())).toBeNull(); + }); + + it('uses ai.description and carries category/objectName/requiresConfirmation', () => { + const def = actionToToolDefinition( + baseAction({ ai: { exposed: true, description: 'Triage a support case and suggest a priority and queue.', category: 'analytics' } }), + undefined, + new Map(), + ); + expect(def).not.toBeNull(); + expect(def!.description).toContain('Triage a support case'); + expect(def!.category).toBe('analytics'); + expect(def!.objectName).toBe('task'); + expect(def!.requiresConfirmation).toBe(false); + }); + + it('summarises ai.outputSchema into the description and carries it through', () => { + const outputSchema = { + type: 'object', + properties: { priority: { type: 'string' }, queue: { type: 'string' } }, + }; + const def = actionToToolDefinition( + baseAction({ ai: { exposed: true, description: 'Triage a support case and return a structured suggestion.', outputSchema } }), + undefined, + new Map(), + ); + expect(def!.outputSchema).toEqual(outputSchema); + expect(def!.description).toMatch(/Returns an object with: priority, queue\./); + }); + + it('merges ai.paramHints into the parameter JSON Schema', () => { + const def = actionToToolDefinition( + baseAction({ + params: [{ name: 'priority', type: 'text' }], + ai: { + exposed: true, + description: 'Set the priority on the task record to one of the allowed values.', + paramHints: { priority: { description: 'One of P0-P3.', enum: ['P0', 'P1', 'P2', 'P3'] } }, + }, + }), + undefined, + new Map(), + ); + const props = (def!.parameters as { properties: Record> }).properties; + expect(props.priority.enum).toEqual(['P0', 'P1', 'P2', 'P3']); + expect(props.priority.description).toBe('One of P0-P3.'); }); }); @@ -372,3 +457,33 @@ describe('actionRequiresApproval + HITL queue routing', () => { expect((result as any).result).toEqual({ deleted: true }); }); }); + +describe('lint guardrail — asserted-safe destructive actions', () => { + it('registers but warns when a destructive action sets ai.requiresConfirmation:false', async () => { + const reg = new ToolRegistry(); + const action = baseAction({ + name: 'archive_task', + mode: 'delete', + type: 'script', + target: 'archiveTaskHandler', + locations: [], + params: [], + ai: { + exposed: true, + description: 'Archive this task record; the operation is reversible from the trash.', + requiresConfirmation: false, + }, + } as Partial); + const objects = [{ name: 'task', label: 'Task', fields: {}, actions: [action] }]; + const { registered, skipped, warnings } = await registerActionsAsTools(reg, { + metadata: { listObjects: async () => objects } as never, + dataEngine: { find: async () => [], executeAction: async () => ({ ok: true }) } as never, + } as never); + + expect(skipped).toEqual([]); + expect(registered).toEqual(['action_archive_task']); + expect(warnings).toHaveLength(1); + expect(warnings[0].action).toBe('archive_task'); + expect(warnings[0].warning).toMatch(/without human approval/); + }); +}); diff --git a/packages/services/service-ai/src/objects/ai-pending-action.object.ts b/packages/services/service-ai/src/objects/ai-pending-action.object.ts index a4696b93a6..f0bf342005 100644 --- a/packages/services/service-ai/src/objects/ai-pending-action.object.ts +++ b/packages/services/service-ai/src/objects/ai-pending-action.object.ts @@ -151,9 +151,9 @@ export const AiPendingActionObject = ObjectSchema.create({ variant: 'primary', confirmText: 'Approve and execute this action now?', successMessage: 'Action approved and executed.', - // The approval click is the operator's authorisation gesture — - // the LLM must not be allowed to bypass HITL by approving itself. - aiExposed: false, + // Human-only by design: not opted into AI (no `ai.exposed`). The approval + // click is the operator's authorisation gesture — the LLM must not be + // able to bypass HITL by approving itself. }, { name: 'reject_pending_action', @@ -165,7 +165,7 @@ export const AiPendingActionObject = ObjectSchema.create({ variant: 'danger', confirmText: 'Reject this pending action? It will not be executed.', successMessage: 'Action rejected.', - aiExposed: false, + // Human-only by design: not opted into AI (no `ai.exposed`). }, ], diff --git a/packages/services/service-ai/src/plugin.ts b/packages/services/service-ai/src/plugin.ts index 6a8f81be9d..d78f6bcce5 100644 --- a/packages/services/service-ai/src/plugin.ts +++ b/packages/services/service-ai/src/plugin.ts @@ -666,7 +666,7 @@ export class AIServicePlugin implements Plugin { const apiBaseUrl = this.options.apiActionBaseUrl ?? process.env.OS_AI_ACTION_API_BASE_URL; const apiHeaders = this.options.apiActionHeaders; - const { registered, skipped } = await registerActionsAsTools( + const { registered, skipped, warnings } = await registerActionsAsTools( this.service.toolRegistry, { metadata: metadataService, @@ -689,6 +689,9 @@ export class AIServicePlugin implements Plugin { { skipped }, ); } + for (const w of warnings) { + ctx.logger.warn(`[AI] action '${w.action}': ${w.warning}`); + } } catch (err) { ctx.logger.warn( '[AI] Failed to register action tools', diff --git a/packages/services/service-ai/src/tools/action-tools.ts b/packages/services/service-ai/src/tools/action-tools.ts index f0c6e9e324..0f5d56a773 100644 --- a/packages/services/service-ai/src/tools/action-tools.ts +++ b/packages/services/service-ai/src/tools/action-tools.ts @@ -1,25 +1,31 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. /** - * Actions-as-Tools — turn declarative {@link Action} metadata into - * AI-callable tools so an agent can not only **read** the user's data + * Actions-as-Tools (ADR-0011) — turn declarative {@link Action} metadata + * into AI-callable tools so an agent can not only **read** the user's data * (via `query_data` / `data_explorer`) but also **act** on it. * - * Phase 1 scope (this module): - * - Only `type: 'script'` actions are auto-exposed. Their handler is - * resolved through {@link IDataEngine.executeAction} (the same - * dispatcher used by Studio's "row toolbar" buttons), so the LLM - * ends up calling exactly the same business logic the UI does. - * - Skip any action that is dangerous (`confirmText`, `variant: 'danger'`, - * `mode: 'delete'`) — these require Phase 2 HITL plumbing. - * - Skip any action whose owner opted out via `aiExposed: false` (the - * hint is read from the action record but not formalised in the Zod - * schema yet; spec change is backwards-compatible additive). + * Exposure is **opt-in** (ADR-0011): an action becomes a tool only when its + * metadata sets `ai.exposed === true`, in which case `ai.description` (the + * LLM-facing contract) is required by the spec. There is no heuristic + * auto-exposure — in an AI-authoring world the opt-in flag is the governance + * gate between "an action exists" and "the agent fleet may invoke it". * - * The tool's JSON Schema is materialised from `action.params[]`, - * resolving field-backed params (`{ field: 'priority' }`) against the - * owning object so the LLM sees the same type/options/required - * constraints the modal dialog would render. + * Supported dispatch types: `script` (via {@link IDataEngine.executeAction} — + * the same dispatcher Studio's row-toolbar buttons use), `api` (HTTP call to + * the action `target`), and `flow` (automation runner). UI-only types + * (`url`, `modal`, `form`) have no headless path and are never exposed. + * + * Destructive actions (`confirmText`, `mode:'delete'`, `variant:'danger'`, + * or `ai.requiresConfirmation:true`) route through the HITL approval queue + * when it is wired (`enableActionApproval` + `aiService`); otherwise they are + * skipped. An author may assert a destructive-looking action is safe by + * setting `ai.requiresConfirmation:false` (the bridge logs a warning). + * + * The tool's JSON Schema is materialised from `action.params[]`, resolving + * field-backed params (`{ field: 'priority' }`) against the owning object so + * the LLM sees the same type/options/required constraints the modal dialog + * would render, then refined by `ai.paramHints`. */ import type { @@ -158,12 +164,28 @@ interface ActionInvocationResult { * headless invocation path. */ /** - * True when an action is "dangerous" and should be routed through the - * HITL approval queue (rather than dispatched directly). Exported so - * Studio's AI-exposure surface can highlight which actions require - * approval. + * True when an AI invocation of this action must be gated behind human + * confirmation (HITL approval queue) rather than dispatched directly. + * + * The author's explicit `ai.requiresConfirmation` wins. When unset, an action + * is treated as confirmation-requiring if it looks destructive (`confirmText` + * set, `mode:'delete'`, or `variant:'danger'`). Exported so Studio's + * AI-exposure surface can highlight which actions require approval. */ export function actionRequiresApproval(action: Action): boolean { + const override = action.ai?.requiresConfirmation; + if (override !== undefined) return override; + return Boolean( + action.confirmText || action.mode === 'delete' || action.variant === 'danger', + ); +} + +/** + * True when an action *looks* destructive by heuristic, independent of any + * `ai.requiresConfirmation` override. Used to detect the "author asserted a + * destructive action is safe" case so the bridge can warn. + */ +function actionLooksDestructive(action: Action): boolean { return Boolean( action.confirmText || action.mode === 'delete' || action.variant === 'danger', ); @@ -179,8 +201,14 @@ export function actionSkipReason( aiService?: ActionToolsContext['aiService']; }, ): string | null { - if (action.aiExposed === false) { - return 'opted-out via aiExposed:false'; + // ADR-0011: opt-in. An action is exposed only when it explicitly says so. + if (action.ai?.exposed !== true) { + return 'not AI-exposed (set ai.exposed:true to opt in)'; + } + // Spec requires a description when exposed; be defensive against raw + // (non-Zod-parsed) metadata reaching this path. + if (!action.ai.description) { + return 'ai.exposed:true but ai.description is missing'; } // Skip Studio-only types (no headless invocation surface). if (action.type === 'url' || action.type === 'modal' || action.type === 'form') { @@ -204,16 +232,23 @@ export function actionSkipReason( return 'no apiClient or apiBaseUrl configured'; } } - // Safety: dangerous actions normally require explicit human approval. - // When the caller has opted in to the HITL queue (and wired aiService), - // we *register* them and route to the queue instead of skipping. + // Safety: actions requiring confirmation must route through the HITL queue. + // When the caller has opted in (and wired aiService), we *register* them and + // route to the queue; otherwise we skip — an exposed action whose author did + // not assert it safe must never run unattended. if (actionRequiresApproval(action)) { const approvalReady = ctx?.enableActionApproval === true && Boolean(ctx?.aiService?.proposePendingAction); if (!approvalReady) { - if (action.confirmText) return 'requires confirmation (confirmText set)'; - if (action.mode === 'delete') return "mode='delete' — destructive"; - if (action.variant === 'danger') return "variant='danger' — destructive"; + const why = + action.ai?.requiresConfirmation === true + ? 'ai.requiresConfirmation:true' + : action.confirmText + ? 'confirmText set' + : action.mode === 'delete' + ? "mode='delete'" + : "variant='danger'"; + return `requires confirmation (${why}) — wire HITL approval (enableActionApproval) or set ai.requiresConfirmation:false to assert it is safe`; } } return null; @@ -341,6 +376,20 @@ function buildParametersSchema( if (resolved.required) required.push(resolved.name); } + // ADR-0011: apply per-parameter AI hints last so they refine the LLM-facing + // schema (tighter enum, clearer description, examples) without touching the + // UI-facing field metadata. Spec validation guarantees keys reference a real + // param (or the injected `recordId`). + const hints = action.ai?.paramHints; + if (hints) { + for (const [key, hint] of Object.entries(hints)) { + const target = properties[key] ?? (properties[key] = { type: 'string' }); + if (hint.description !== undefined) target.description = hint.description; + if (hint.enum !== undefined) target.enum = hint.enum; + if (hint.examples !== undefined) target.examples = hint.examples; + } + } + return { type: 'object', properties, @@ -356,23 +405,40 @@ export function actionToolName(action: Action, prefix = 'action_'): string { return `${prefix}${action.name}`; } -function describeAction(action: Action, ownerObject: ObjectDef | undefined): string { +/** + * Defensive fallback description for raw (non-Zod-parsed) metadata that + * somehow reaches the bridge with `ai.exposed:true` but no `ai.description`. + * Normal authored actions never hit this — the spec requires a description + * when exposed. We deliberately do NOT derive descriptions from the UI label + * for the happy path (ADR-0011 Non-Goal). + */ +function fallbackDescription(action: Action, ownerObject: ObjectDef | undefined): string { const label = - typeof action.label === 'string' - ? action.label - : action.name.replace(/_/g, ' '); - const target = action.objectName ?? ownerObject?.name; - const targetLabel = ownerObject?.label ?? target; - const parts: string[] = []; - parts.push(`${label}${targetLabel ? ` — operates on ${targetLabel}` : ''}.`); - if (action.successMessage && typeof action.successMessage === 'string') { - parts.push(`On success: ${action.successMessage}`); + typeof action.label === 'string' ? action.label : action.name.replace(/_/g, ' '); + const targetLabel = ownerObject?.label ?? action.objectName ?? ownerObject?.name; + return `${label}${targetLabel ? ` — operates on ${targetLabel}` : ''}.`; +} + +/** Top-level property names of a JSON-Schema object, if any. */ +function outputSchemaKeys(schema: Record | undefined): string[] { + if (!schema || typeof schema !== 'object') return []; + const props = (schema as { properties?: unknown }).properties; + if (!props || typeof props !== 'object') return []; + return Object.keys(props as Record); +} + +/** + * Compose the LLM-facing tool description: the authored `ai.description` + * (required when exposed), plus a compact "Returns:" line summarising + * `ai.outputSchema` so the model can reason about chaining the result. + */ +function buildToolDescription(action: Action, ownerObject: ObjectDef | undefined): string { + const base = action.ai?.description ?? fallbackDescription(action, ownerObject); + const keys = outputSchemaKeys(action.ai?.outputSchema); + if (keys.length > 0) { + return `${base}\n\nReturns an object with: ${keys.join(', ')}.`; } - if (action.mode) parts.push(`Mode: ${action.mode}.`); - parts.push( - 'Use this when the user asks to perform this operation in natural language.', - ); - return parts.join(' '); + return base; } // ── Builders ──────────────────────────────────────────────────────── @@ -392,12 +458,16 @@ export function actionToToolDefinition( // with full context (apiClient, automation, HITL approval wiring). This // function only checks the *structural* invariants that make a // definition impossible to build. - if (action.aiExposed === false) return null; + if (action.ai?.exposed !== true) return null; if (action.type === 'url' || action.type === 'modal' || action.type === 'form') return null; return { name: actionToolName(action, toolPrefix), - description: describeAction(action, ownerObject), + description: buildToolDescription(action, ownerObject), parameters: buildParametersSchema(action, ownerObject, allObjects), + ...(action.ai.category ? { category: action.ai.category } : {}), + ...(action.ai.outputSchema ? { outputSchema: action.ai.outputSchema } : {}), + ...(action.objectName ? { objectName: action.objectName } : {}), + requiresConfirmation: actionRequiresApproval(action), }; } @@ -763,6 +833,12 @@ export async function registerActionsAsTools( ): Promise<{ registered: string[]; skipped: Array<{ action: string; reason: string }>; + /** + * Non-fatal lint advisories surfaced while building tools — e.g. an author + * exposed a destructive-looking action and asserted it safe via + * `ai.requiresConfirmation:false`. Callers (the plugin) log these. + */ + warnings: Array<{ action: string; warning: string }>; }> { const objects = (await context.metadata.listObjects()) as ObjectDef[]; const objMap = new Map( @@ -771,6 +847,7 @@ export async function registerActionsAsTools( const registered: string[] = []; const skipped: Array<{ action: string; reason: string }> = []; + const warnings: Array<{ action: string; warning: string }> = []; const prefix = context.toolPrefix ?? 'action_'; for (const obj of objects) { @@ -796,6 +873,18 @@ export async function registerActionsAsTools( continue; } + // Lint guardrail: the action is exposed and will run unattended, yet it + // looks destructive and the author explicitly asserted it safe. Register + // it (the author's call) but make the assertion visible. + if (actionLooksDestructive(normalized) && normalized.ai?.requiresConfirmation === false) { + warnings.push({ + action: normalized.name, + warning: + 'exposed destructive-looking action with ai.requiresConfirmation:false — ' + + 'it will run without human approval; confirm this is intended.', + }); + } + const definition = actionToToolDefinition(normalized, obj, objMap, prefix); if (!definition) continue; @@ -857,5 +946,5 @@ export async function registerActionsAsTools( } } - return { registered, skipped }; + return { registered, skipped, warnings }; } diff --git a/packages/spec/src/contracts/ai-service.ts b/packages/spec/src/contracts/ai-service.ts index 9e3736d07a..c5045d235f 100644 --- a/packages/spec/src/contracts/ai-service.ts +++ b/packages/spec/src/contracts/ai-service.ts @@ -170,6 +170,25 @@ export interface AIToolDefinition { description: string; /** JSON Schema describing the tool parameters */ parameters: Record; + /** + * Optional tool category (mirrors `ToolSchema.category`). Carried by + * action-backed tools from `action.ai.category`; surfaced by tool-listing + * routes. Not sent to the model. + */ + category?: string; + /** + * Optional JSON Schema for the tool's return value. Action-backed tools + * derive this from `action.ai.outputSchema` to enable downstream chaining. + */ + outputSchema?: Record; + /** Object this tool operates on, when the tool is action-backed. */ + objectName?: string; + /** + * Whether invoking this tool requires human-in-the-loop confirmation. + * Action-backed tools set this from the action's confirmation policy + * (`action.ai.requiresConfirmation`, or the destructive-action default). + */ + requiresConfirmation?: boolean; } // --------------------------------------------------------------------------- diff --git a/packages/spec/src/ui/action.form.ts b/packages/spec/src/ui/action.form.ts index cd5fa044fe..fe0e17199b 100644 --- a/packages/spec/src/ui/action.form.ts +++ b/packages/spec/src/ui/action.form.ts @@ -58,7 +58,7 @@ export const actionForm = defineForm({ columns: 2, fields: [ { field: 'bulkEnabled', colSpan: 1, helpText: 'Allow applying to multiple selected records' }, - { field: 'aiExposed', colSpan: 1, helpText: 'Allow AI agents to call this action' }, + { field: 'ai', colSpan: 2, helpText: 'AI exposure (opt-in): set ai.exposed=true and write ai.description (≥40 chars) to make this callable by agents.' }, { field: 'recordIdParam', visibleOn: "data.type == 'api'", colSpan: 1, helpText: 'Body parameter name for record ID' }, { field: 'recordIdField', visibleOn: "data.type == 'api' && data.recordIdParam", colSpan: 1, helpText: 'Field to use as record ID (default: "id")' }, { field: 'bodyShape', visibleOn: "data.type == 'api'", colSpan: 2, helpText: 'Request body structure (flat or nested)' }, diff --git a/packages/spec/src/ui/action.test.ts b/packages/spec/src/ui/action.test.ts index 23b34a9104..3cb6547d88 100644 --- a/packages/spec/src/ui/action.test.ts +++ b/packages/spec/src/ui/action.test.ts @@ -564,6 +564,108 @@ describe('Action I18n Integration', () => { }); }); +// ============================================================================ +// ADR-0011: AI exposure block (opt-in) +// ============================================================================ + +describe('ActionSchema - ai block (ADR-0011)', () => { + const longDescription = + 'Classify a support case and suggest a priority, category, and queue for the agent.'; + + it('defaults ai.exposed to false when an ai block is supplied without it', () => { + const result = ActionSchema.parse({ + name: 'maybe_expose', + label: 'Maybe', + ai: {}, + }); + expect(result.ai?.exposed).toBe(false); + }); + + it('accepts an action with no ai block (not exposed)', () => { + const result = ActionSchema.parse({ name: 'plain', label: 'Plain' }); + expect(result.ai).toBeUndefined(); + }); + + it('requires a description when exposed is true', () => { + expect(() => + ActionSchema.parse({ + name: 'expose_no_desc', + label: 'Expose', + ai: { exposed: true }, + }), + ).toThrow(/ai\.description/); + }); + + it('rejects a description shorter than 40 chars', () => { + expect(() => + ActionSchema.parse({ + name: 'expose_short', + label: 'Expose', + ai: { exposed: true, description: 'too short' }, + }), + ).toThrow(); + }); + + it('accepts a fully-specified ai block', () => { + const result = ActionSchema.parse({ + name: 'triage_case', + label: 'Triage Case', + objectName: 'crm_case', + params: [{ name: 'priority', type: 'text' }], + ai: { + exposed: true, + description: longDescription, + category: 'analytics', + paramHints: { priority: { description: 'P0-P3', enum: ['P0', 'P1', 'P2', 'P3'] } }, + outputSchema: { type: 'object', properties: { priority: { type: 'string' } } }, + requiresConfirmation: false, + }, + }); + expect(result.ai?.exposed).toBe(true); + expect(result.ai?.category).toBe('analytics'); + expect(result.ai?.requiresConfirmation).toBe(false); + }); + + it('rejects an invalid ai.category value', () => { + expect(() => + ActionSchema.parse({ + name: 'bad_category', + label: 'Bad', + ai: { exposed: true, description: longDescription, category: 'not_a_category' }, + }), + ).toThrow(); + }); + + it('rejects paramHints keys that do not match a declared param', () => { + expect(() => + ActionSchema.parse({ + name: 'bad_hint', + label: 'Bad Hint', + params: [{ name: 'priority', type: 'text' }], + ai: { exposed: true, description: longDescription, paramHints: { nonexistent: { description: 'x' } } }, + }), + ).toThrow(/paramHints/); + }); + + it('allows paramHints to reference the injected recordId', () => { + expect(() => + ActionSchema.parse({ + name: 'hint_record_id', + label: 'Hint', + objectName: 'task', + locations: ['record_header'], + ai: { exposed: true, description: longDescription, paramHints: { recordId: { description: 'The task id.' } } }, + }), + ).not.toThrow(); + }); + + it('does not require a description when exposed is false', () => { + expect(() => + ActionSchema.parse({ name: 'opted_out', label: 'Out', ai: { exposed: false } }), + ).not.toThrow(); + }); +}); + describe('Action ARIA Integration', () => { it('should accept action with ARIA attributes', () => { expect(() => ActionSchema.parse({ diff --git a/packages/spec/src/ui/action.zod.ts b/packages/spec/src/ui/action.zod.ts index 92dea71f96..3b7f4cd8d0 100644 --- a/packages/spec/src/ui/action.zod.ts +++ b/packages/spec/src/ui/action.zod.ts @@ -149,6 +149,90 @@ export const ACTION_LOCATIONS = [ export const ActionLocationSchema = z.enum(ACTION_LOCATIONS); export type ActionLocation = z.infer; +/** + * Tool category values for {@link ActionAiSchema.category}. + * + * Mirrors `ToolCategorySchema` in `../ai/tool.zod`. Kept **inline** rather + * than imported to avoid a `ui → ai` import cycle (`ai/*.form.ts` already + * imports `defineForm` from `ui/view.zod`). If you change the canonical + * tool categories, update both sides. + */ +const ActionAiCategorySchema = z.enum([ + 'data', + 'action', + 'flow', + 'integration', + 'vector_search', + 'analytics', + 'utility', +]); + +/** + * AI exposure block (ADR-0011 "Actions as AI Tools"). + * + * **Opt-in, default off.** An action becomes an AI-callable tool only when + * `exposed: true`. This is a deliberate governance gate: in an AI-authoring + * world the platform's value is that a human can govern exactly which + * capabilities the agent fleet is allowed to invoke — a half-finished or + * unreviewed action must never be silently armed. + * + * When exposed, `description` is **required** — it is the LLM-facing contract + * (when/why to call), authored explicitly rather than derived from the + * UI `label`. The bridge in `@objectstack/service-ai` translates this block + * into an `AIToolDefinition`. + */ +export const ActionAiSchema = z.object({ + /** + * Expose this action to AI agents as a callable tool. Default `false`. + * Setting `true` REQUIRES `description`. + */ + exposed: z.boolean().default(false).describe('Expose this action to AI agents. Requires `description` when true.'), + + /** + * LLM-facing description: tells the model when and why to call this action. + * Distinct from the UI `label`. Plain English, ≥ 40 chars for useful tool + * selection. Required whenever `exposed` is true. + */ + description: z.string().min(40).optional().describe('LLM-facing description (≥40 chars). Required when exposed.'), + + /** + * Override the derived tool category. Defaults to `action` (side-effect). + * Use `data` for read-only actions, `analytics` for aggregations, etc. + */ + category: ActionAiCategorySchema.optional().describe('Tool category override (defaults to "action").'), + + /** + * Per-parameter AI hints, keyed by param name (or the injected `recordId`). + * Tightens the JSON Schema the LLM sees (e.g. add `enum`, override + * `description`, supply `examples`) WITHOUT changing the UI-facing field + * metadata. Keys must match a declared `params[].name` (or `recordId`). + */ + paramHints: z.record(z.string(), z.object({ + description: z.string().optional(), + enum: z.array(z.union([z.string(), z.number()])).optional(), + examples: z.array(z.unknown()).optional(), + })).optional().describe('Per-parameter AI hints keyed by param name.'), + + /** + * Output JSON Schema for the action's return value. Enables structured + * downstream tool chaining (one action's output feeds another's input) and + * is summarised into the tool description so the model knows what it gets + * back. Optional — when omitted the return value is treated as freeform. + */ + outputSchema: z.record(z.string(), z.unknown()).optional().describe('JSON Schema for the action return value.'), + + /** + * Override confirmation for AI calls. When unset, the bridge defaults to + * `true` for actions that look destructive (`confirmText` set, `mode:'delete'`, + * or `variant:'danger'`). Set explicitly to `false` to assert a destructive- + * looking action is safe to run without human approval, or `true` to force a + * human-in-the-loop gate on an otherwise-safe action. + */ + requiresConfirmation: z.boolean().optional().describe('Override HITL confirmation for AI invocations.'), +}); + +export type ActionAi = z.infer; + export const ActionSchema = lazySchema(() => z.object({ /** Machine name of the action */ name: SnakeCaseIdentifierSchema.describe('Machine name (lowercase snake_case)'), @@ -289,13 +373,11 @@ export const ActionSchema = lazySchema(() => z.object({ bulkEnabled: z.boolean().optional().describe('Whether this action can be applied to multiple selected records'), /** - * AI exposure opt-out. When `false`, this action is **not** auto-registered - * as an AI tool (see service-ai's `registerActionsAsTools`). When unset or - * `true` the platform decides based on safety heuristics (script-type, no - * confirmation gate, not destructive). Use this for sensitive ops you want - * to keep human-only even though they otherwise look safe. + * AI exposure block (ADR-0011). Opt-in, default off: an action is exposed + * to AI agents only when `ai.exposed === true`, in which case `ai.description` + * is required. See {@link ActionAiSchema}. */ - aiExposed: z.boolean().optional().describe('Set to false to keep this action out of the AI tool registry.'), + ai: ActionAiSchema.optional().describe('AI exposure (opt-in). Set ai.exposed=true + ai.description to make this callable by agents.'), /** * Row-context: when the action runs from a list_item location, this body key @@ -359,6 +441,29 @@ export const ActionSchema = lazySchema(() => z.object({ }, { message: "Action 'target' is required when type is 'url', 'flow', 'modal', 'api', or 'form'.", path: ['target'], +}).refine((data) => { + // ADR-0011: an exposed action must carry an LLM-facing description. + if (data.ai?.exposed === true && !data.ai.description) { + return false; + } + return true; +}, { + message: 'ai.description is required (≥40 chars) when ai.exposed is true.', + path: ['ai', 'description'], +}).refine((data) => { + // ADR-0011: paramHints keys must reference a declared param (or the + // auto-injected `recordId`), so a typo can't silently no-op. + const hints = data.ai?.paramHints; + if (!hints) return true; + const known = new Set(['recordId']); + for (const p of data.params ?? []) { + const key = p.name ?? p.field; + if (key) known.add(key); + } + return Object.keys(hints).every((k) => known.has(k)); +}, { + message: 'ai.paramHints keys must match a declared param name (or "recordId").', + path: ['ai', 'paramHints'], })); export type Action = z.infer;