Uh oh!
There was an error while loading. Please reload this page.
docs(liveness): stop counting authoring/preview renderers as runtime consumers (#1878) - #3685
Merged
Merged
Conversation
…consumers (#1878) A 2026-06 sweep that "missed objectui" over-corrected and marked 13 properties `live` on the evidence of a metadata-admin previews/*Preview.tsx panel — a surface that only echoes what the author typed. `live` is supposed to mean "authoring this changes runtime behaviour". Three of the 13 were re-verified in the §3 naming-drift recheck; all three were wrong or misleading: - skill.permissions -> dead + authorWarn. The cloud SkillRegistry reads only active/triggerConditions/tools; nothing gates invocation by this list. Mirrors the identical tool.permissions entry. (Cloud documents the intent in a comment without implementing it.) - agent.knowledge -> dead + authorWarn. search_knowledge takes sourceIds from the LLM's tool-call arguments, never from the agent record, so a declared knowledge scope constrains nothing (and an agent can search sources it was never scoped to). - action.disabled: verdict kept (it IS live now) but evidence corrected to the six rendering surfaces. At ledger-writing time it was enforced on ONE of six, so the preview-based verdict hid a real five-surface silent no-op — fixed in objectui#2863. Also: sync the hand-maintained count table (agent 14/5/0 -> 13/5/1, skill 10/-/- -> 9/-/1) and add a "an authoring/preview renderer is NOT a runtime consumer" section naming the ten still-unverified preview-only claims, with the tie-break rule: when in doubt prefer dead + authorWarn — a warning on a property that works costs nothing; silence on a property that does nothing ships a bug. check:liveness green (exit 0); the 13 unregistered-proof warnings are pre-existing on main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 104 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Jul 27, 2026
xuyushun441-sys pushed a commit
that referenced
this pull request
Jul 27, 2026
…f 10 were wrong (#3686) Closes the sweep opened in #3685. Final tally across all 13 properties the 2026-06 pass marked `live` on preview-renderer evidence: 3 stand, 10 were wrong — a 77% error rate for that standard. Corrected to dead + authorWarn (with actionable hints): - action.shortcut / action.bulkEnabled — registered into ActionEngine, but getShortcuts()/handleShortcut()/getBulkActions()/executeBulk() have no non-test caller and no keydown listener feeds them. Bulk toolbars are driven by the LIST VIEW's bulkActions/bulkActionDefs instead. - flow.active — deprecated no-op; `status` is what gates binding/execution. Sharper than inert: the spec default is false while an unset flow runs, so the key reads as if it disabled the flow. - skill.triggerPhrases — a runtime path reads it, but only to hand it back to API clients. Phrases are never matched against user messages; activation is triggerConditions + the agent's skills[] + explicit /skill-name pinning. - tool.category / active / builtIn — not part of AIToolDefinition; the tool set reaches the LLM as name/description/parameters only. Contrast agent.active and skill.active, which ARE enforced. - tool.requiresConfirmation — SAFETY-shaped and unenforced on every path (LLM tool set, ToolRegistry.execute, the REST execute route, the MCP bridge). Every real requiresConfirmation read is action.ai.* — a different, live property. ADR-0033 already resolved to delete this placeholder. Kept live, evidence corrected to the real reader: - action.execute — ActionRunner.ts:704 + the spec's execute->target transform. Records a live divergence: the transform prefers `target`, ActionRunner prefers `execute`, so an action declaring both runs different code on the two sides. - flow.status — engine.ts:1374-1382 gates binding AND execution since 497bda8. The file-level _note claiming "status/active gate nothing" was true when written and became false a month later; rewritten. README: count tables synced (action 33/1/2, flow 26/-/5, skill 8/-/2, tool 5/1/5) and the methodology section extended with the final tally, the two failure directions (over- AND under-stating liveness), and the three search traps that produced false negatives during this work — including that macOS `git grep -E` silently ignores `\b`. check:liveness green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
os-zhuang added a commit
that referenced
this pull request
Jul 27, 2026
…f 10 were wrong (#3686) (#3711) Closes the sweep opened in #3685. Final tally across all 13 properties the 2026-06 pass marked `live` on preview-renderer evidence: 3 stand, 10 were wrong — a 77% error rate for that standard. Corrected to dead + authorWarn (with actionable hints): - action.shortcut / action.bulkEnabled — registered into ActionEngine, but getShortcuts()/handleShortcut()/getBulkActions()/executeBulk() have no non-test caller and no keydown listener feeds them. Bulk toolbars are driven by the LIST VIEW's bulkActions/bulkActionDefs instead. - flow.active — deprecated no-op; `status` is what gates binding/execution. Sharper than inert: the spec default is false while an unset flow runs, so the key reads as if it disabled the flow. - skill.triggerPhrases — a runtime path reads it, but only to hand it back to API clients. Phrases are never matched against user messages; activation is triggerConditions + the agent's skills[] + explicit /skill-name pinning. - tool.category / active / builtIn — not part of AIToolDefinition; the tool set reaches the LLM as name/description/parameters only. Contrast agent.active and skill.active, which ARE enforced. - tool.requiresConfirmation — SAFETY-shaped and unenforced on every path (LLM tool set, ToolRegistry.execute, the REST execute route, the MCP bridge). Every real requiresConfirmation read is action.ai.* — a different, live property. ADR-0033 already resolved to delete this placeholder. Kept live, evidence corrected to the real reader: - action.execute — ActionRunner.ts:704 + the spec's execute->target transform. Records a live divergence: the transform prefers `target`, ActionRunner prefers `execute`, so an action declaring both runs different code on the two sides. - flow.status — engine.ts:1374-1382 gates binding AND execution since 497bda8. The file-level _note claiming "status/active gate nothing" was true when written and became false a month later; rewritten. README: count tables synced (action 33/1/2, flow 26/-/5, skill 8/-/2, tool 5/1/5) and the methodology section extended with the final tally, the two failure directions (over- AND under-stating liveness), and the three search traps that produced false negatives during this work — including that macOS `git grep -E` silently ignores `\b`. check:liveness green. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-through of the §3 naming-drift recheck (#3677). Ledger-only; releases nothing.
The calibration bug
liveis supposed to mean authoring the property changes runtime behaviour. But a 2026-06 sweep that "missed objectui" over-corrected and marked 13 propertiesliveciting only ametadata-admin/previews/*Preview.tsxpanel — a surface that echoes back what the author typed. It proves the property round-trips; it proves nothing about enforcement.Three of those 13 were re-verified during the recheck. All three were wrong or misleading:
skill.permissionslive(preview)SkillRegistryreads onlyactive/triggerConditions/tools— nothing gates invocation by this listdead+authorWarnagent.knowledgelive(preview)search_knowledgetakessourceIdsfrom the LLM's tool-call args, never from the agent record — a declared scope constrains nothingdead+authorWarnaction.disabledlive(preview)live, evidence correctedThat last row is the sharpest evidence: the preview-based verdict was technically "live" while a spec-authored
disableddid nothing on five of six surfaces (fixed in objectui#2863, dogfooded by #3643).Changes
skill.json/agent.json— corrected todead+authorWarn+ an actionableauthorHint(gate at the agent / restrict at the knowledge service).skill.permissionsnow mirrors the identicaltool.permissionsentry, which was alreadydead+authorWarn— the two were giving authors opposite signals for the same non-enforcement.action.json— evidence updated to the six real rendering surfaces, with the history recorded.liveness/README.md— count table synced (hand-maintained: agent14/5/0→13/5/1, skill10/–/–→9/–/1) and a new "an authoring/preview renderer is NOT a runtime consumer" section that names the ten still-unverified preview-only claims (action.execute/shortcut/bulkEnabled,flow.status/active,skill.triggerPhrases,tool.category/requiresConfirmation/active/builtIn) and states the tie-break rule: when in doubt preferdead+authorWarn— a warning on a property that works costs nothing; silence on a property that does nothing ships a bug.What this does NOT do
It does not decide whether skill-level permissions or agent knowledge-scoping should be enforced — both runtimes live in the closed cloud repo, and cloud even documents the skill-permission intent in a comment without implementing it. That enforce-or-prune decision is filed separately; this PR only stops the ledger from telling authors a security-shaped field is live when it isn't.
Refs #1878, #3677.
🤖 Generated with Claude Code