From ba76fb7d7b5fc689963cf78543e4e25c5a6c9ef9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 16:44:21 +0000 Subject: [PATCH 1/3] docs(spec): re-close the tool.json liveness ledger against the real cloud runtime Every entry cited `packages/services/service-ai/...`, a path that exists in neither repo: the framework has no service-ai tree at all, and cloud's real layout is `packages/service-ai/...`. The citations stayed green only because the gate hardcodes exactly that wrong spelling in FOREIGN_PATH_PREFIXES and so never resolved them. Re-closed against cloud origin/main@15f55df. `name` and `description` gain framework-local ANCHORED evidence (the packages/mcp bridge reads both off each AIToolDefinition), so CI can falsify them from this checkout for the first time. `label` gains its first evidence pointer ever. `parameters` and `outputSchema` are confirmed with their negative halves measured rather than asserted. `objectName` is left `live` and deliberately UNDATED: the cloud walk falsified its stated basis, but the only remaining candidate reader is an objectui read-back that this container could not walk, and publishing a negative cross-repo verdict without that walk is the app.homePageId failure shape. The file-level `_note` is repaired: its "stale build artifact with no src/" claim is now simply false (the tree is absent outright), its cited path prefix was wrong for cloud too, and its "the OPEN framework edition does not consume them" claim was false for two keys. No verdict moved, so state-counts.md is untouched and still current. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KX8wnyjStaZcuMyAMNsy3N --- .changeset/tool-liveness-ledger-reclose.md | 46 ++++++++++++++++++++++ packages/spec/liveness/tool.json | 38 ++++++++++++------ 2 files changed, 72 insertions(+), 12 deletions(-) create mode 100644 .changeset/tool-liveness-ledger-reclose.md diff --git a/.changeset/tool-liveness-ledger-reclose.md b/.changeset/tool-liveness-ledger-reclose.md new file mode 100644 index 0000000000..084b89f77f --- /dev/null +++ b/.changeset/tool-liveness-ledger-reclose.md @@ -0,0 +1,46 @@ +--- +'@objectstack/spec': patch +--- + +liveness ledger: re-close `tool.json` against the real cloud runtime, and repair a `_note` that was false three ways (#13042) + +All six entries cited `packages/services/service-ai/…` — a path that exists in +**neither** repo. The framework has no service-ai tree at all (`git ls-files` +matches 0 paths containing it; `packages/services/` holds 16 members, none of +them service-ai), and the cloud repo's real layout is `packages/service-ai/…` +and `packages/service-ai-studio/…`. Six pointers into thin air, green for as +long as they existed because `scripts/liveness/evidence.mts` hardcodes exactly +the wrong spelling in `FOREIGN_PATH_PREFIXES` and so never resolved them. + +Re-closed against cloud `origin/main@15f55df` — the first pass run from a +container with a cloud checkout in reach, which is the executor constraint that +parked this card. + +- `name`, `description` — confirmed `live`, and both now carry **framework-local + anchored** evidence (`packages/mcp/…#registerToolFromDefinition`, the MCP + bridge that reads each `AIToolDefinition`) beside the cloud citation. CI can + falsify them from this checkout for the first time; the old note's claim that + "the OPEN framework edition does not consume them" was false for these two. +- `parameters` — confirmed `live` on the cloud LLM path. Deliberately *not* + co-cited to the MCP bridge: that bridge never forwards the key, while its own + docblock says it does (filed as #13271). +- `label` — confirmed `live` and given its **first evidence pointer ever**; it + had carried a bare `live` with no `evidence` field since seeding, the one row + the #13003 census could not even call stale. +- `outputSchema` — stays `experimental`, with the negative half now measured + rather than asserted: the key occurs on exactly four non-test lines in the + whole cloud repo, and none of them validates anything. +- `objectName` — **left `live` and deliberately UNDATED.** The cloud walk + falsified its stated basis (both cited sites read `action.objectName`, and + `action.json` carries the identical citation verbatim; the same-named + `AIToolDefinition.objectName` is written and read by nothing). It is not + re-graded here because the only remaining candidate reader is an objectui + read-back, and no objectui checkout was reachable — publishing `dead` on that + search is the `app.homePageId` failure shape. Staying undated keeps it on the + `--stale-verification` worklist, which is the honest place for a claim that + could not be closed. + +Data-only: no schema, no runtime, no authoring surface changes, and no verdict +moved, so `state-counts.md` is untouched and still current. `liveness/` is in +this package's `files` array, so these ledgers ship in the npm tarball and this +is published data. diff --git a/packages/spec/liveness/tool.json b/packages/spec/liveness/tool.json index 3df8be3398..4f3e3de92a 100644 --- a/packages/spec/liveness/tool.json +++ b/packages/spec/liveness/tool.json @@ -1,35 +1,49 @@ { "type": "tool", - "_note": "ToolSchema. Seeded from docs/audits/2026-06-toolschema-property-liveness.md. Tool metadata is WRITE-ONLY (projected one-way for Studio; runtime uses a separate AIToolDefinition). LIVE props are live via that same-named surface, not metadata read-back. ⚠ EVIDENCE LIVES IN CLOUD/EE: the `packages/services/service-ai/...` paths cited below are the closed `@objectstack/service-ai` runtime in the CLOUD repo, NOT git-tracked framework code (the framework's own service-ai tree is a stale build artifact with no src/). These props are `live` because that cloud runtime consumes them; the OPEN framework edition does not — see content/docs/ai for the open/cloud boundary. 2026-07-30 (#3896 close-out): the four inert authoring keys (category/permissions/active/builtIn) were REMOVED from ToolSchema — permissions promised an invocation gate nothing enforced, active:false withdrew nothing; strict-rejected with prescriptions (TOOL_RETIRED_KEY_GUIDANCE) and stripped by the tool-inert-authoring-keys-removed conversion. Entries deleted per the requiresConfirmation precedent (#3715).", + "_note": "ToolSchema. Seeded from docs/audits/2026-06-toolschema-property-liveness.md. Tool metadata is WRITE-ONLY (projected one-way for Studio; runtime uses a separate AIToolDefinition). LIVE props are live via that same-named surface, not metadata read-back. ⚠ EVIDENCE LIVES MOSTLY IN CLOUD: every `cloud`-attributed path below is the closed `@objectstack/service-ai` / `@objectstack/service-ai-studio` runtime in the CLOUD repo, not git-tracked here — see content/docs/ai for the open/cloud boundary. 2026-08-29 RE-CLOSED against cloud `origin/main@15f55df`, the first pass run from a container that actually had a cloud checkout in reach; three things the old note asserted turned out to be false, and each failed in its own way. (a) It said the framework's own service-ai tree 'is a stale build artifact with no src/'. That was stale twice over and is now simply FALSE — the tree is ABSENT: `git ls-files` matches 0 paths containing service-ai, no such directory exists on disk, and `packages/services/` holds 16 members, none of them service-ai. There is no artifact left to be stale. (b) The cited path prefix was wrong for the CLOUD repo too. Cloud's real layout is `packages/service-ai/…` and `packages/service-ai-studio/…`, never `packages/services/service-ai/…`, so the five citations this file used to carry named a path that exists in NEITHER repo — unfalsifiable pointers, which is exactly what the README says a `live` verdict must never rest on. ⚠ MECHANICAL TRAP for anyone repointing a sibling ledger: `scripts/liveness/evidence.mts` hardcodes only `packages/services/service-ai/` in FOREIGN_PATH_PREFIXES, which is why the WRONG spelling sat green for so long; the REAL cloud path is repo-rooted in shape and is NOT in that list, so an unmarked repoint resolves as LOCAL and fails CI. Every citation of it MUST carry the `cloud` realm marker. The stale spelling and this note's sentence (a) both survive in action.json (3 entries), agent.json (11) and skill.json (8) — filed as #13272. (c) 'the OPEN framework edition does not consume them' is FALSE for `name` and `description`: `packages/mcp` bridges each AIToolDefinition onto an MCP server and reads both. Those two entries now carry framework-local ANCHORED evidence, so CI can falsify them from this checkout — the condition #13042 was filed for. 2026-07-30 (#3896 close-out): the four inert authoring keys (category/permissions/active/builtIn) were REMOVED from ToolSchema — permissions promised an invocation gate nothing enforced, active:false withdrew nothing; strict-rejected with prescriptions (TOOL_RETIRED_KEY_GUIDANCE) and stripped by the tool-inert-authoring-keys-removed conversion. Entries deleted per the requiresConfirmation precedent (#3715).", "props": { "name": { "status": "live", - "evidence": "packages/services/service-ai/src/adapters/vercel-adapter.ts:46", - "note": "selection + LLM function key." + "verifiedAt": "2026-08-29", + "evidenceScope": "cross-repo", + "evidence": "packages/mcp/src/mcp-server-runtime.ts#registerToolFromDefinition registers every bridged tool on the MCP server under that name and routes each incoming call back to the registry by the same name; cloud @15f55df: packages/service-ai/src/tools/tool-registry.ts#register keys the definition map by it and packages/service-ai/src/adapters/vercel-adapter.ts#buildVercelOptions keys the LLM tool set by it", + "producer": "packages/mcp/src/plugin.ts#start duck-types the AI service's toolRegistry before calling bridgeTools, so the framework-side reader sees a tool only when a cloud AI service is mounted; cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#actionToToolDefinition mints each action-backed definition and packages/service-ai-studio/src/plugin.ts#init registers the shipped platform ones", + "note": "selection + LLM function key. Re-closed 2026-08-29 against cloud @15f55df, and read at BOTH ends of the same-named surface this file's `_note` describes: the framework MCP bridge and the cloud LLM adapter each key on it, and the cloud registry stores definitions under it — which is also why a name collision silently replaces a tool rather than erroring." }, "label": { "status": "live", - "note": "display." + "verifiedAt": "2026-08-29", + "evidenceScope": "cross-repo", + "evidence": "cloud @15f55df: packages/service-ai-studio/src/plugin.ts#toToolLabel — the boot-time Studio ingest inside AIStudioPlugin.init reads the label off each AIToolDefinition, falls back to a name-derived one when unset, and persists the result as the tool metadata record; that call is the ONLY site in the whole cloud repo that registers tool metadata", + "producer": "cloud @15f55df: packages/service-ai/src/tools/data-tools.ts and packages/service-ai-studio/src/tools/apply-edit.tool.ts are representative of the shipped definition literals that set it. Cited for REACHABILITY rather than for a missing supplier — the read point runs unconditionally at plugin boot, and its toToolLabel fallback means even an unset value still changes what is persisted", + "note": "display, and it IS displayed — the #7131 display-key rule, met at the projection rather than at a preview. RE-CLOSED 2026-08-29 (cloud @15f55df) and given its FIRST evidence pointer: this entry had carried a bare `live` with no `evidence` field at all since seeding, which is why it was the one row #13042's census could not even call stale — a claim with nothing to rot. ⚠ Scope of the verdict: the read is the WRITE side of the one-way Studio projection, i.e. cloud reading the label off the AIToolDefinition to compose the persisted record. Whether objectui then RENDERS that record's label was not walked (no objectui checkout in the re-closing container). That limit does not weaken a positive claim — one real reader is enough — but it does mean this row says nothing about the read-back side." }, "description": { "status": "live", - "evidence": "packages/services/service-ai/src/adapters/vercel-adapter.ts:46", - "note": "sent to the LLM." + "verifiedAt": "2026-08-29", + "evidenceScope": "cross-repo", + "evidence": "packages/mcp/src/mcp-server-runtime.ts#registerToolFromDefinition passes it through verbatim as the registered MCP tool's description; cloud @15f55df: packages/service-ai/src/adapters/vercel-adapter.ts#buildVercelOptions is what puts it in front of the model, and packages/service-ai/src/tools/action-tools.ts#buildToolDescription composes it for action-backed tools", + "producer": "packages/mcp/src/plugin.ts#start gates the framework-side reader on a mounted AI service exposing a toolRegistry; cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#buildToolDescription supplies it for action-backed tools and packages/service-ai/src/tools/action-tools.ts#fallbackDescription covers an exposed action that authored none", + "note": "sent to the LLM. Re-closed 2026-08-29 against cloud @15f55df — this is the string the model reads to decide whether to call the tool, so it is behavioural rather than display, and the framework MCP bridge reuses the same string verbatim for MCP clients." }, "parameters": { "status": "live", - "evidence": "packages/services/service-ai/src/adapters/vercel-adapter.ts:46", - "note": "LLM function schema." + "verifiedAt": "2026-08-29", + "evidenceScope": "cross-repo", + "evidence": "cloud @15f55df: packages/service-ai/src/adapters/vercel-adapter.ts#buildVercelOptions wraps it with the SDK jsonSchema helper and passes it as the tool's inputSchema, which is what constrains the arguments the model may emit", + "note": "LLM function schema. Re-closed 2026-08-29 against cloud @15f55df. ⚠ Deliberately NOT co-cited to the framework MCP bridge, and the asymmetry with `name` / `description` is the finding: registerToolFromDefinition registers each bridged tool with NO inputSchema, and the annotations it does pass carry only destructiveHint / readOnlyHint / openWorldHint — so this key never reaches an MCP client, while its own docblock claims the JSON Schema is passed as annotations metadata. Filed as #13271. The verdict rests on the cloud LLM path alone, where it is unambiguously live." }, "objectName": { "status": "live", - "evidence": "packages/services/service-ai/src/tools/action-tools.ts:535", - "note": "action-tool dispatch." + "evidence": "cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#actionToToolDefinition copies the action's objectName onto the AIToolDefinition field of the same name, and packages/service-ai/src/tools/action-tools.ts#createActionToolHandler refuses to dispatch without one", + "note": "action-tool dispatch. ⚠ VERDICT NOT RE-CLOSED — deliberately left UNDATED after the 2026-08-29 cloud walk (@15f55df), so it stays on the `--stale-verification` worklist instead of being stamped on evidence that does not support it. What the walk found: BOTH cited sites read `action.objectName`, the ActionSchema key — whose own row in action.json carries the very same citation, `packages/services/service-ai/src/tools/action-tools.ts:535`, word for word. The same-named surface this file's `_note` rests the type's verdicts on, `AIToolDefinition.objectName`, is WRITTEN by actionToToolDefinition and read by nothing: zero readers in cloud, zero in this repo. Nor does it reach persisted metadata — the single `metadataService.register` tool site in cloud ingests only the shipped Studio definitions, and none of those is action-backed. So the basis for `live` is unsupported for THIS key. It is NOT re-graded from here for one reason: the only remaining candidate reader is an objectui read-back of a persisted tool record, and no objectui checkout was reachable in the re-closing container. Publishing `dead` on a negative cross-repo claim without that walk is the #4895 / app.homePageId failure shape exactly — the verdict that was wrong because the search could not see the renderer. Needs a seat with objectui; a `dead` re-grade would also move the tool row in state-counts.md, which #13042's file fence excluded." }, "outputSchema": { "status": "experimental", - "evidence": "packages/services/service-ai/src/tools/action-tools.ts:437", - "note": "keys folded into description only; no validation exists." + "verifiedAt": "2026-08-29", + "evidenceScope": "cross-repo", + "evidence": "cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#outputSchemaKeys lists the top-level property names and packages/service-ai/src/tools/action-tools.ts#buildToolDescription folds them into the LLM-facing description as a trailing Returns-an-object-with line", + "note": "keys folded into description only; no validation exists. Re-closed 2026-08-29 against cloud @15f55df and BOTH halves hold. The fold is real, at the two anchored sites. The negative half is now measured rather than asserted: `outputSchema` occurs on exactly four non-test lines in the entire cloud repo — the two folding sites, the docblock naming them, and the spread that copies action.ai.outputSchema onto the AIToolDefinition field, which nothing reads back. No validator, no chaining consumer, in cloud or in this repo. ⚠ Two scope notes. Both folding sites read `action.ai.outputSchema`, not this key, so what actually carries the verdict is the spec's own `[EXPERIMENTAL — not enforced]` describe marker, which is a legal resolution source in its own right and does not depend on a consumer at all. And objectui was not walked; that is immaterial here, since a renderer cannot be where output validation would live." } } } From aab65accf11f7bcda520c99d6155bf2e466337d7 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 16:52:33 +0000 Subject: [PATCH 2/3] docs(spec): point the tool.objectName ledger note at its follow-up card The entry records a measured falsification that this container could not act on (the re-grade needs an objectui reader). The measurement now has its own card, so the note names it rather than leaving the reader to re-derive it. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KX8wnyjStaZcuMyAMNsy3N --- packages/spec/liveness/tool.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/spec/liveness/tool.json b/packages/spec/liveness/tool.json index 4f3e3de92a..bef22696f2 100644 --- a/packages/spec/liveness/tool.json +++ b/packages/spec/liveness/tool.json @@ -36,7 +36,7 @@ "objectName": { "status": "live", "evidence": "cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#actionToToolDefinition copies the action's objectName onto the AIToolDefinition field of the same name, and packages/service-ai/src/tools/action-tools.ts#createActionToolHandler refuses to dispatch without one", - "note": "action-tool dispatch. ⚠ VERDICT NOT RE-CLOSED — deliberately left UNDATED after the 2026-08-29 cloud walk (@15f55df), so it stays on the `--stale-verification` worklist instead of being stamped on evidence that does not support it. What the walk found: BOTH cited sites read `action.objectName`, the ActionSchema key — whose own row in action.json carries the very same citation, `packages/services/service-ai/src/tools/action-tools.ts:535`, word for word. The same-named surface this file's `_note` rests the type's verdicts on, `AIToolDefinition.objectName`, is WRITTEN by actionToToolDefinition and read by nothing: zero readers in cloud, zero in this repo. Nor does it reach persisted metadata — the single `metadataService.register` tool site in cloud ingests only the shipped Studio definitions, and none of those is action-backed. So the basis for `live` is unsupported for THIS key. It is NOT re-graded from here for one reason: the only remaining candidate reader is an objectui read-back of a persisted tool record, and no objectui checkout was reachable in the re-closing container. Publishing `dead` on a negative cross-repo claim without that walk is the #4895 / app.homePageId failure shape exactly — the verdict that was wrong because the search could not see the renderer. Needs a seat with objectui; a `dead` re-grade would also move the tool row in state-counts.md, which #13042's file fence excluded." + "note": "action-tool dispatch. ⚠ VERDICT NOT RE-CLOSED — deliberately left UNDATED after the 2026-08-29 cloud walk (@15f55df), so it stays on the `--stale-verification` worklist instead of being stamped on evidence that does not support it. What the walk found: BOTH cited sites read `action.objectName`, the ActionSchema key — whose own row in action.json carries the very same citation, `packages/services/service-ai/src/tools/action-tools.ts:535`, word for word. The same-named surface this file's `_note` rests the type's verdicts on, `AIToolDefinition.objectName`, is WRITTEN by actionToToolDefinition and read by nothing: zero readers in cloud, zero in this repo. Nor does it reach persisted metadata — the single `metadataService.register` tool site in cloud ingests only the shipped Studio definitions, and none of those is action-backed. So the basis for `live` is unsupported for THIS key. It is NOT re-graded from here for one reason: the only remaining candidate reader is an objectui read-back of a persisted tool record, and no objectui checkout was reachable in the re-closing container. Publishing `dead` on a negative cross-repo claim without that walk is the #4895 / app.homePageId failure shape exactly — the verdict that was wrong because the search could not see the renderer. Needs a seat with objectui — filed as #13276, which carries the full measurement so the next reader starts from it; a `dead` re-grade would also move the tool row in state-counts.md, which #13042's file fence excluded." }, "outputSchema": { "status": "experimental", From 4bc1737c26f782cb5a4ea4c99acf3401127e22e5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 17:15:46 +0000 Subject: [PATCH 3/3] docs(spec): close tool.objectName against objectui, and correct the note's write-only claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An objectui checkout came into reach, so the one entry the cloud walk could not settle is now settled — and the answer inverts the reasoning rather than the verdict. `objectName` stays `live`, but not for the reason the row gave: its "action-tool dispatch" basis is falsified (both cited sites read the ActionSchema key, and AIToolDefinition.objectName is written and read by nothing), so the key binds and routes nothing. The consumer that does exist is objectui's registered metadata-admin preview, which reads it off the persisted record and renders it as the header object pill — the #7131 display-key rule, pinned in ToolPreview.test.tsx over a comment reading "objectName is NOT residue". Live as a DISPLAY key, and explicitly not an ADR-0049 retirement candidate. That reading also falsifies the _note's fourth claim, "tool metadata is WRITE-ONLY ... not metadata read-back" — which is precisely why nobody had looked at the renderer. ToolPreview reads all six props off the stored record. The type has two live bases (the AIToolDefinition surface, behavioural; and the metadata read-back, display) and every row now names the one it rests on. The stale "objectui not walked" hedges on `label` and `outputSchema` are corrected in the same pass. All six entries are now dated. No verdict moved, so state-counts.md stays untouched. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01KX8wnyjStaZcuMyAMNsy3N --- .changeset/tool-liveness-ledger-reclose.md | 33 ++++++++++++++-------- packages/spec/liveness/tool.json | 15 ++++++---- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/.changeset/tool-liveness-ledger-reclose.md b/.changeset/tool-liveness-ledger-reclose.md index 084b89f77f..c517747fc3 100644 --- a/.changeset/tool-liveness-ledger-reclose.md +++ b/.changeset/tool-liveness-ledger-reclose.md @@ -12,9 +12,9 @@ and `packages/service-ai-studio/…`. Six pointers into thin air, green for as long as they existed because `scripts/liveness/evidence.mts` hardcodes exactly the wrong spelling in `FOREIGN_PATH_PREFIXES` and so never resolved them. -Re-closed against cloud `origin/main@15f55df` — the first pass run from a -container with a cloud checkout in reach, which is the executor constraint that -parked this card. +Re-closed against cloud `origin/main@15f55df` and objectui `origin/main@26896c6` +— the first pass run from a container with both checkouts in reach, which is the +executor constraint that parked this card. All six entries are now dated. - `name`, `description` — confirmed `live`, and both now carry **framework-local anchored** evidence (`packages/mcp/…#registerToolFromDefinition`, the MCP @@ -30,15 +30,24 @@ parked this card. - `outputSchema` — stays `experimental`, with the negative half now measured rather than asserted: the key occurs on exactly four non-test lines in the whole cloud repo, and none of them validates anything. -- `objectName` — **left `live` and deliberately UNDATED.** The cloud walk - falsified its stated basis (both cited sites read `action.objectName`, and - `action.json` carries the identical citation verbatim; the same-named - `AIToolDefinition.objectName` is written and read by nothing). It is not - re-graded here because the only remaining candidate reader is an objectui - read-back, and no objectui checkout was reachable — publishing `dead` on that - search is the `app.homePageId` failure shape. Staying undated keeps it on the - `--stale-verification` worklist, which is the honest place for a claim that - could not be closed. +- `objectName` — stays `live`, but **on a completely different basis**, and the + row now says so. Its stated basis is falsified: both cited sites read + `action.objectName` (`action.json` carries the identical citation verbatim), + and the same-named `AIToolDefinition.objectName` is written and read by + nothing, so the key gates, binds and routes nothing. The consumer that does + exist is objectui's registered metadata-admin preview, which reads it off the + persisted record and renders it as the header's object pill — the #7131 + display-key rule, and pinned in `ToolPreview.test.tsx` over a comment reading + "`objectName` is NOT residue". So: `live` as a **display** key, never as a + binding, and explicitly not an ADR-0049 retirement candidate — retiring it + would delete a key the renderer deliberately renders and tests. + +The `_note`'s fourth false claim is corrected in the same pass: "tool metadata +is WRITE-ONLY … not metadata read-back" is simply untrue, and it is the reason +nobody had looked at the renderer. `ToolPreview` reads **all six** props off the +stored record. The type therefore has two live bases — the `AIToolDefinition` +surface (behavioural) and the metadata read-back (display) — and each row now +names which one carries it. Data-only: no schema, no runtime, no authoring surface changes, and no verdict moved, so `state-counts.md` is untouched and still current. `liveness/` is in diff --git a/packages/spec/liveness/tool.json b/packages/spec/liveness/tool.json index bef22696f2..08067ead80 100644 --- a/packages/spec/liveness/tool.json +++ b/packages/spec/liveness/tool.json @@ -1,6 +1,6 @@ { "type": "tool", - "_note": "ToolSchema. Seeded from docs/audits/2026-06-toolschema-property-liveness.md. Tool metadata is WRITE-ONLY (projected one-way for Studio; runtime uses a separate AIToolDefinition). LIVE props are live via that same-named surface, not metadata read-back. ⚠ EVIDENCE LIVES MOSTLY IN CLOUD: every `cloud`-attributed path below is the closed `@objectstack/service-ai` / `@objectstack/service-ai-studio` runtime in the CLOUD repo, not git-tracked here — see content/docs/ai for the open/cloud boundary. 2026-08-29 RE-CLOSED against cloud `origin/main@15f55df`, the first pass run from a container that actually had a cloud checkout in reach; three things the old note asserted turned out to be false, and each failed in its own way. (a) It said the framework's own service-ai tree 'is a stale build artifact with no src/'. That was stale twice over and is now simply FALSE — the tree is ABSENT: `git ls-files` matches 0 paths containing service-ai, no such directory exists on disk, and `packages/services/` holds 16 members, none of them service-ai. There is no artifact left to be stale. (b) The cited path prefix was wrong for the CLOUD repo too. Cloud's real layout is `packages/service-ai/…` and `packages/service-ai-studio/…`, never `packages/services/service-ai/…`, so the five citations this file used to carry named a path that exists in NEITHER repo — unfalsifiable pointers, which is exactly what the README says a `live` verdict must never rest on. ⚠ MECHANICAL TRAP for anyone repointing a sibling ledger: `scripts/liveness/evidence.mts` hardcodes only `packages/services/service-ai/` in FOREIGN_PATH_PREFIXES, which is why the WRONG spelling sat green for so long; the REAL cloud path is repo-rooted in shape and is NOT in that list, so an unmarked repoint resolves as LOCAL and fails CI. Every citation of it MUST carry the `cloud` realm marker. The stale spelling and this note's sentence (a) both survive in action.json (3 entries), agent.json (11) and skill.json (8) — filed as #13272. (c) 'the OPEN framework edition does not consume them' is FALSE for `name` and `description`: `packages/mcp` bridges each AIToolDefinition onto an MCP server and reads both. Those two entries now carry framework-local ANCHORED evidence, so CI can falsify them from this checkout — the condition #13042 was filed for. 2026-07-30 (#3896 close-out): the four inert authoring keys (category/permissions/active/builtIn) were REMOVED from ToolSchema — permissions promised an invocation gate nothing enforced, active:false withdrew nothing; strict-rejected with prescriptions (TOOL_RETIRED_KEY_GUIDANCE) and stripped by the tool-inert-authoring-keys-removed conversion. Entries deleted per the requiresConfirmation precedent (#3715).", + "_note": "ToolSchema. Seeded from docs/audits/2026-06-toolschema-property-liveness.md. Tool metadata is not an execution entry point — the RUNTIME uses a separate AIToolDefinition, and most props here are live via that same-named surface. ⚠ 2026-08-29: the framing this note carried for years, 'WRITE-ONLY … not metadata read-back', is FALSIFIED and was actively harmful — it is why nobody looked at the renderer. objectui's metadata-admin preview IS a registered consumer of persisted `tool` records (objectui @26896c6, `registerMetadataPreview('tool', ToolPreview)`), and ToolPreview reads ALL SIX props off the stored record: name, label, description, objectName, parameters, outputSchema. So there are TWO live bases on this type, and each row says which one it rests on: the AIToolDefinition surface (behavioural) and the metadata read-back (display, the #7131 rule). `objectName` is the row that turns on the difference — it has NO behavioural reader anywhere and is live on the read-back alone. ⚠ EVIDENCE LIVES MOSTLY IN CLOUD: every `cloud`-attributed path below is the closed `@objectstack/service-ai` / `@objectstack/service-ai-studio` runtime in the CLOUD repo, not git-tracked here — see content/docs/ai for the open/cloud boundary. 2026-08-29 RE-CLOSED against cloud `origin/main@15f55df`, the first pass run from a container that actually had a cloud checkout in reach; three things the old note asserted turned out to be false, and each failed in its own way. (a) It said the framework's own service-ai tree 'is a stale build artifact with no src/'. That was stale twice over and is now simply FALSE — the tree is ABSENT: `git ls-files` matches 0 paths containing service-ai, no such directory exists on disk, and `packages/services/` holds 16 members, none of them service-ai. There is no artifact left to be stale. (b) The cited path prefix was wrong for the CLOUD repo too. Cloud's real layout is `packages/service-ai/…` and `packages/service-ai-studio/…`, never `packages/services/service-ai/…`, so the five citations this file used to carry named a path that exists in NEITHER repo — unfalsifiable pointers, which is exactly what the README says a `live` verdict must never rest on. ⚠ MECHANICAL TRAP for anyone repointing a sibling ledger: `scripts/liveness/evidence.mts` hardcodes only `packages/services/service-ai/` in FOREIGN_PATH_PREFIXES, which is why the WRONG spelling sat green for so long; the REAL cloud path is repo-rooted in shape and is NOT in that list, so an unmarked repoint resolves as LOCAL and fails CI. Every citation of it MUST carry the `cloud` realm marker. The stale spelling and this note's sentence (a) both survive in action.json (3 entries), agent.json (11) and skill.json (8) — filed as #13272. (c) 'the OPEN framework edition does not consume them' is FALSE for `name` and `description`: `packages/mcp` bridges each AIToolDefinition onto an MCP server and reads both. Those two entries now carry framework-local ANCHORED evidence, so CI can falsify them from this checkout — the condition #13042 was filed for. (d) the write-only claim above, corrected in the same pass once an objectui checkout came into reach. Note the pattern across all four: every one of them was a NEGATIVE claim ('no src', 'not framework code', 'the open edition does not', 'not read back') that no gate could falsify, and three of the four were wrong. The realms named in each `evidenceScope` are the realms actually walked — cloud @15f55df and objectui @26896c6 — and never 'everywhere'. 2026-07-30 (#3896 close-out): the four inert authoring keys (category/permissions/active/builtIn) were REMOVED from ToolSchema — permissions promised an invocation gate nothing enforced, active:false withdrew nothing; strict-rejected with prescriptions (TOOL_RETIRED_KEY_GUIDANCE) and stripped by the tool-inert-authoring-keys-removed conversion. Entries deleted per the requiresConfirmation precedent (#3715).", "props": { "name": { "status": "live", @@ -14,9 +14,9 @@ "status": "live", "verifiedAt": "2026-08-29", "evidenceScope": "cross-repo", - "evidence": "cloud @15f55df: packages/service-ai-studio/src/plugin.ts#toToolLabel — the boot-time Studio ingest inside AIStudioPlugin.init reads the label off each AIToolDefinition, falls back to a name-derived one when unset, and persists the result as the tool metadata record; that call is the ONLY site in the whole cloud repo that registers tool metadata", + "evidence": "cloud @15f55df: packages/service-ai-studio/src/plugin.ts#toToolLabel — the boot-time Studio ingest inside AIStudioPlugin.init reads the label off each AIToolDefinition, falls back to a name-derived one when unset, and persists the result as the tool metadata record; that call is the ONLY site in the whole cloud repo that registers tool metadata. objectui @26896c6: packages/app-shell/src/views/metadata-admin/previews/ToolPreview.tsx#ToolPreview then reads that persisted label back and renders it as the preview header, with the tool's machine name as fallback", "producer": "cloud @15f55df: packages/service-ai/src/tools/data-tools.ts and packages/service-ai-studio/src/tools/apply-edit.tool.ts are representative of the shipped definition literals that set it. Cited for REACHABILITY rather than for a missing supplier — the read point runs unconditionally at plugin boot, and its toToolLabel fallback means even an unset value still changes what is persisted", - "note": "display, and it IS displayed — the #7131 display-key rule, met at the projection rather than at a preview. RE-CLOSED 2026-08-29 (cloud @15f55df) and given its FIRST evidence pointer: this entry had carried a bare `live` with no `evidence` field at all since seeding, which is why it was the one row #13042's census could not even call stale — a claim with nothing to rot. ⚠ Scope of the verdict: the read is the WRITE side of the one-way Studio projection, i.e. cloud reading the label off the AIToolDefinition to compose the persisted record. Whether objectui then RENDERS that record's label was not walked (no objectui checkout in the re-closing container). That limit does not weaken a positive claim — one real reader is enough — but it does mean this row says nothing about the read-back side." + "note": "display, and it IS displayed — the #7131 display-key rule, met at the projection rather than at a preview. RE-CLOSED 2026-08-29 (cloud @15f55df) and given its FIRST evidence pointer: this entry had carried a bare `live` with no `evidence` field at all since seeding, which is why it was the one row #13042's census could not even call stale — a claim with nothing to rot. Both sides of the round trip are now cited: cloud reads the label off the AIToolDefinition to compose the persisted record, and objectui's registered metadata-admin preview reads that record back and renders it. (An earlier pass of this re-closure recorded the read-back as un-walked for want of an objectui checkout; the walk has since been taken, and this row no longer carries that hedge.)" }, "description": { "status": "live", @@ -35,15 +35,18 @@ }, "objectName": { "status": "live", - "evidence": "cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#actionToToolDefinition copies the action's objectName onto the AIToolDefinition field of the same name, and packages/service-ai/src/tools/action-tools.ts#createActionToolHandler refuses to dispatch without one", - "note": "action-tool dispatch. ⚠ VERDICT NOT RE-CLOSED — deliberately left UNDATED after the 2026-08-29 cloud walk (@15f55df), so it stays on the `--stale-verification` worklist instead of being stamped on evidence that does not support it. What the walk found: BOTH cited sites read `action.objectName`, the ActionSchema key — whose own row in action.json carries the very same citation, `packages/services/service-ai/src/tools/action-tools.ts:535`, word for word. The same-named surface this file's `_note` rests the type's verdicts on, `AIToolDefinition.objectName`, is WRITTEN by actionToToolDefinition and read by nothing: zero readers in cloud, zero in this repo. Nor does it reach persisted metadata — the single `metadataService.register` tool site in cloud ingests only the shipped Studio definitions, and none of those is action-backed. So the basis for `live` is unsupported for THIS key. It is NOT re-graded from here for one reason: the only remaining candidate reader is an objectui read-back of a persisted tool record, and no objectui checkout was reachable in the re-closing container. Publishing `dead` on a negative cross-repo claim without that walk is the #4895 / app.homePageId failure shape exactly — the verdict that was wrong because the search could not see the renderer. Needs a seat with objectui — filed as #13276, which carries the full measurement so the next reader starts from it; a `dead` re-grade would also move the tool row in state-counts.md, which #13042's file fence excluded." + "verifiedAt": "2026-08-29", + "evidenceScope": "cross-repo", + "evidence": "objectui @26896c6: packages/app-shell/src/views/metadata-admin/previews/ToolPreview.tsx#ToolPreview takes objectName off the persisted tool record and renders it as the header's object pill, and drops that pill row entirely when the key is absent", + "producer": "objectui @26896c6: packages/app-shell/src/views/metadata-admin/previews/index.ts#registerMetadataPreview binds ToolPreview to the `tool` type and packages/app-shell/src/views/metadata-admin/ResourceEditPage.tsx#getMetadataPreview resolves that registration and hands the component the draft being edited — the read point is reachable, not merely present. The author IS the supplier of the value itself", + "note": "⚠ RE-CLOSED 2026-08-29, and the verdict SURVIVES for a reason that is NOT the one this row used to give — read both halves, because each is load-bearing. (1) The old basis is FALSIFIED. The row claimed 'action-tool dispatch' citing cloud action-tools.ts. Both sites there read `action.objectName`, the ActionSchema key, whose own row in action.json carries that very citation word for word; the same-named `AIToolDefinition.objectName` is WRITTEN by actionToToolDefinition and read by nothing — zero readers in cloud @15f55df, zero in this repo — and it never reaches persisted metadata either, since cloud's single `metadataService.register` tool site ingests only the shipped Studio definitions, none action-backed. So this key gates, binds and routes NOTHING; anyone reading the old note as a dispatch guarantee was misled. (2) A real consumer exists anyway, on the OTHER basis: objectui's registered metadata-admin preview reads it off the persisted record and renders it. That is the #7131 display-key rule — the render is not standing in for a behaviour, because for authored tool metadata there is no execution layer for it to stand in for: ToolSchema's own describe says a metadata-authored tool is a read-only projection that no executor loads. objectui pins this deliberately — ToolPreview.test.tsx asserts the pill renders and disappears with the key, over a comment reading 'objectName is NOT residue — ToolSchema still accepts it'. ⇒ `live` as a DISPLAY key, never as a binding. ⛔ Do NOT read this row as licence to retire the key under ADR-0049: a `dead` verdict here would retire a key the renderer deliberately renders and tests, the `requiresService` / `colorVariant` mistake from the other end. The prior pass left this undated rather than grading `dead` precisely because it could not see objectui; that caution was correct — the renderer is where the answer was." }, "outputSchema": { "status": "experimental", "verifiedAt": "2026-08-29", "evidenceScope": "cross-repo", "evidence": "cloud @15f55df: packages/service-ai/src/tools/action-tools.ts#outputSchemaKeys lists the top-level property names and packages/service-ai/src/tools/action-tools.ts#buildToolDescription folds them into the LLM-facing description as a trailing Returns-an-object-with line", - "note": "keys folded into description only; no validation exists. Re-closed 2026-08-29 against cloud @15f55df and BOTH halves hold. The fold is real, at the two anchored sites. The negative half is now measured rather than asserted: `outputSchema` occurs on exactly four non-test lines in the entire cloud repo — the two folding sites, the docblock naming them, and the spread that copies action.ai.outputSchema onto the AIToolDefinition field, which nothing reads back. No validator, no chaining consumer, in cloud or in this repo. ⚠ Two scope notes. Both folding sites read `action.ai.outputSchema`, not this key, so what actually carries the verdict is the spec's own `[EXPERIMENTAL — not enforced]` describe marker, which is a legal resolution source in its own right and does not depend on a consumer at all. And objectui was not walked; that is immaterial here, since a renderer cannot be where output validation would live." + "note": "keys folded into description only; no validation exists. Re-closed 2026-08-29 against cloud @15f55df and BOTH halves hold. The fold is real, at the two anchored sites. The negative half is now measured rather than asserted: `outputSchema` occurs on exactly four non-test lines in the entire cloud repo — the two folding sites, the docblock naming them, and the spread that copies action.ai.outputSchema onto the AIToolDefinition field, which nothing reads back. No validator, no chaining consumer, in cloud or in this repo. ⚠ Two scope notes. Both folding sites read `action.ai.outputSchema`, not this key, so what actually carries the verdict is the spec's own `[EXPERIMENTAL — not enforced]` describe marker, which is a legal resolution source in its own right and does not depend on a consumer at all. And objectui HAS since been walked: ToolPreview reads the persisted record's outputSchema and renders it as an 'Output Schema' mirror table. That is a display read, not enforcement — it does not disturb `experimental`, which is a claim about the absence of VALIDATION, and a renderer is not where validation would live." } } }